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Symbols & Abbreviations 


ADS: Automatic Dependent Surveillance 

ADS-B: Automatic Dependent Surveillance-Broadcast 

ADS-C: Automatic Dependent Surveillance-Contract 

AS AS: Airborne Separation Assistance System 

ASTOR: Aircraft Simulation for Traffic Operations Research 

ATC: Air Traffic Control 
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Mode Control Panel 
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N: 

Sample size 

NASA: 

National Aeronautics and Space Administration 

NATOTS: 
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Navigation Display 

NextGen: 

Next Generation Air Transportation System 

NLR: 

National Aerospace Laboratory of The Netherlands 

nm: 
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Operational Services and Environment Description 
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XCP: Transponder Control Panel 
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Definitions 


Automatic Dependent Surveillance-Broadcast (ADS-B). A system by which airplanes constantly 
broadcast their current position and altitude, category of aircraft, ground speed, flight number, and 
whether the aircraft is turning, climbing or descending over a dedicated radio datalink. 

Automatic Dependent Surveillance-Contract (ADS-C). ADS-C enables appropriately equipped aircraft to 
send position information messages at predetermined geographical locations, at specified time intervals or 
at the occurrence of specified events. ADS-C can be relayed via SATCOM data link, or VHF data link. 

Flight Level A surface of constant atmospheric pressure, which is related to a specific pressure datum of 
29.92 inches of mercury (1013.2 hectopascals), and is separated from other such surfaces by specific 
pressure intervals. Each is stated in three digits that represent hundreds of feet (ft). For example, flight 
level (FL) 370 represents a barometric altimeter indication of 37,000 ft. 

Following Climb or Descent . A Same Track climb or descent performed by an aircraft when following a 
reference aircraft. 

Ground Speed Differential The difference between the In-Trail Procedure (ITP) Aircraft’s ground speed 
and a Reference or Same Direction Potentially Blocking Aircraft’s ground speed. 

Initial Flight Level The flight level of the ITP Aircraft when it determines a climb or descent is desired. 

Intermediate Flight Level Any Flight Level between the Requested Flight Level and the Initial Flight 
Level of the ITP Aircraft. 

Intervening Flight Level Any Intermediate Flight Level that has Same Direction aircraft whose ADS-B 
report data are available to the ITP Aircraft. 

In-Trail Procedure (ITP). A procedure employed by an aircraft that desires to change its flight level to a 
new flight level by climbing or descending in front or behind one or two Same Track, Potentially 
Blocking Aircraft which are at an Intervening Flight Level. 

ITP Aircraft. An aircraft that is fully qualified (from an equipment, operator, and flight crew qualification 
standpoint) to conduct an ITP and whose flight crew is considering a change of flight level. 

ITP Criteria. A set of conditions that must be satisfied prior to initiating or executing an ITP clearance. 

ITP Distance. The distance between Reference or Potentially Blocking Aircraft and the ITP Aircraft as 
defined by the difference in distance to a common point along each aircraft’s track. For the special case 
of parallel tracks, an along-track distance measurement would be used to determine this value. 

ITP Separation Minimum. The longitudinal separation minimum between the ITP Aircraft and Reference 
Aircraft. The ITP Separation Minimum is based on the International Civil Aviation Organization (ICAO) 
Distance Measuring Equipment (DME) separation method and is 10 nautical miles (nm). 

ITP Equipment. Equipment needed on the ITP Aircraft that provides ADS-B information on Potentially 
Blocking Aircraft with regard to ADS-B data qualification (i.e., information sufficient to determine if 



ADS-B data are, or are not, Qualified ADS-B Data), Same Direction, ITP Distance, Ground Speed 
Differential, flight level, and aircraft identification. 

ITP Speed/Distance Criteria. A specified set of maximum Positive Ground Speed Differential and 
minimum ITP Distance values between a Same Direction Potentially Blocking Aircraft and the ITP 
Aircraft, required to be met prior to requesting or initiating an ITP with that aircraft as a Reference 
Aircraft. 

Leading Climb or Descent. A Same Track climb or descent performed by an aircraft when ahead of a 
reference aircraft. 

Positive Ground Speed Differential. A Ground Speed Differential value which would cause the ITP 
Distance between the ITP Aircraft and the Reference Aircraft to decrease. 

Potentially Blocking Aircraft. Aircraft at an Intervening Flight Level whose ADS-B report data are 
available to the ITP Aircraft. 

Positive Mach Difference. A difference in Mach between the ITP Aircraft and the Reference Aircraft that 
would result in a decrease in the ITP Distance between them. 

Qualified ADS-B Data. Received ADS-B data that meet the accuracy and integrity requirements 
determined to be required for the ITP. 

Reference Aircraft. One or two Same Direction, Potentially Blocking Aircraft with Qualified ADS-B Data 
that meet the ITP Speed/Distance Criteria and that will be identified to ATC by the ITP Aircraft as part of 
the ITP clearance request. 

Requested Flight Level. A flight level above (for a climb) or below (for a descent) all Intervening Flight 
Levels that is no more than 4,000 ft from the Initial Flight Level. 

Same Direction. Same direction tracks and intersecting tracks or portions thereof, the angular difference 
of which is less than 45 degrees or more than 315 degrees. 

Same Track. Same direction tracks and intersecting tracks or portions thereof, the angular difference of 
which is less than 45 degrees or more than 315 degrees, and whose protection areas overlap (i.e., without 
lateral separation). 



Abstract 


The Enhanced Oceanic Operations Human-In-The-Loop In-Trail Procedure 
(ITP) Validation Simulation Study investigated the viability of an ITP designed 
to enable oceanic flight level changes that would not otherwise be possible. 

Twelve commercial airline pilots with current oceanic experience flew a series 
of simulated scenarios involving either standard or ITP flight level change 
maneuvers and provided subjective workload ratings, assessments of ITP 
validity and acceptability, and objective performance measures associated with 
the appropriate selection, request, and execution of ITP flight level change 
manuevers. In the majority of scenarios, subject pilots correctly assessed the 
traffic situation, selected an appropriate response (i.e., either a standard flight 
level change request, an ITP request, or no request), and executed their 
selected flight level change procedure, if any, without error. Workload ratings 
for ITP maneuvers were acceptable and not substantially higher than for 
standard flight level change maneuvers, and, for the majority of scenarios and 
subject pilots, subjective acceptability ratings and comments for ITP were 
generally high and positive. Qualitatively, the ITP was found to be valid and 
acceptable. However, the error rates for ITP maneuvers were higher than for 
standard flight level changes, and these errors may have design implications 
for both the ITP and the study's prototype traffic display. These errors and 
their implications are discussed. 

Introduction 

The Human-In-The-Loop (HITL) In-Trail Procedure (ITP) Validation Simulation Study was conducted in 
September 2006 under sponsorship of the Enhanced Oceanic Operations (EOO) research element, which is part 
of the National Aeronautics and Space Administration (NASA) Next Generation Air Transportation System 
(NextGen) Air Traffic Management (ATM) Airspace Project. EOO’s objective is to develop a globally 
accepted, early application of airborne Automatic Dependent Surveillance-Broadcast (ADS-B) that results in 
more efficient oceanic and remote non-radar operations while providing opportunities for research of Airborne 
Separation Assistance Systems (AS AS). The early application should provide operational experience with 
ASAS and be an incentive for operators to voluntarily equip their aircraft with transformational technologies. 

The oceanic and remote non-radar airspace was selected as a proving ground for researching ASAS concepts 
since these domains already contain key characteristics of NextGen. They provide performance-based services 
where the individual equipage level impacts separation services, and aircraft use trajectory -based operations that 
include time management for separation and traffic flow management. 

Aircraft in oceanic and remote non-radar airspace frequently fly for extended periods of time in the same 
direction and along similar flight paths as other aircraft. These flight paths, or tracks, are typically either 
defined by the authority controlling the airspace or requested by the respective aircraft, but in either case are 
designed to optimize flight safety and efficiency. The track definitions are additionally constrained by the 
requirement, for air traffic control (ATC) coordination, to connect at each end with a relatively small number of 
defined waypoints in the respective continental flight information regions. The end result of these track design 
constraints is that for practical considerations, the enormous expanse of airspace is reduced to only a few 
“optimal” tracks for use by many of the transport-category aircraft, with only one or two of these tracks 
considered “best” for any given flight. 
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Since there is no radar surveillance and only limited direct communications available in oceanic and remote 
airspace, controllers use procedural separation rules to ensure that aircraft remain separated. Procedural 
separation typically requires much greater times or distances between aircraft than when radar surveillance and 
direct communications are available. For example, in the North Atlantic Organized Track System (NATOTS) 
(Figure 1), the tracks are spaced no closer than 60 nautical miles (nm) from each other to allow for the position 
uncertainties involved with aircraft joining or leaving the track system [1]. This lateral spacing requirement 
limits the total number of tracks. Along a given track, aircraft must be spaced at track entry so that even with 
their different flight-planned airspeeds, they will transit and exit the track separated by at least 10 minutes in 
time, which is approximately 80 nm in distance, at typical transport category aircraft cruise speeds with no 
closure rate. This longitudinal spacing requirement limits the number of aircraft on a given track at a given 
altitude or flight level. 



Figure 1. North Atlantic Organized Track System 


Depending on the specific airspace rules in effect, controllers can separate aircraft vertically by as little as 
1,000 feet (ft) (e.g., at multiple flight levels on a track) which somewhat alleviates the traffic density limits 
imposed by lateral and longitudinal procedural separation requirements. However, not all flight levels are 
practical for all aircraft. Some flight levels may be unusable due to an unacceptable level of turbulence or 
headwinds. Also, it is important that long-range aircraft fly near their most fuel-efficient altitudes. Flying 
several thousand feet higher or lower than the optimal altitude can result in thousands of pounds of additional 
fuel being burned during an oceanic crossing, and in some cases can even result in a diversion to a closer airport 
due to insufficient fuel to complete the originally-planned flight. 

An aircraft’s most fuel-efficient altitude is lower during the early segments of a flight, when the aircraft has a 
heavy fuel load, and higher during the later segments when much of the fuel has been consumed. Hence, 
operational efficiency is generally enhanced by climbing to a higher flight level one or more times during a long 
flight segment. There are also occasionally times when descent to a lower flight level is desirable, usually to 
avoid turbulence or unfavorable winds. However, in many situations the standard longitudinal separation does 
not exist at one or more higher or lower flight levels, but does exist at a succeeding flight level. An aircraft 
desiring a flight level change to such a succeeding flight level would therefore be prevented or “blocked” from 
making the climb or descent through the intervening flight level(s). For example, in Figure 2, the trailing 
aircraft at Flight Level (FL) 340 (shown in blue) desires a climb to FL360 but cannot be cleared for the climb 
because there would be insufficient longitudinal separation with the aircraft at FL350 (shown in red) during the 
climb. 
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Figure 2. Aircraft at Flight Level 340 (FL340) desires a climb to FL360 


Researchers at NASA LaRC, along with others in the worldwide ATM research and development 
community, have been developing an ITP concept which would increase the opportunities for such flight level 
changes that would otherwise be blocked. The ITP would employ new onboard avionics equipment that would 
provide an equipped aircraft’s crew with improved information about nearby traffic and would introduce new 
procedures that would enable the crew, when appropriate criteria are met, to request an ITP flight level change 
referencing one or two of these nearby aircraft that might otherwise block the flight level change. The ITP 
equipment would use a surveillance technology known as ADS-B that broadcasts aircraft position and other 
essential data via an onboard transponder. ADS-B data received from nearby aircraft would yield more accurate 
position data than that available to controllers of oceanic/remote airspace, enabling controllers to approve ITP 
flight level change requests that reference these aircraft even if standard separation criteria would not otherwise 
exist with these reference aircraft. That is, the availability of more accurate airborne surveillance data would 
enable safe flight level changes through intervening flight levels, with lower separation minima than when using 
current ground-based non-radar separation rules. It should be noted that the ITP concept departs from 
conventional ground-based ATM practices because it would more actively involve flight crews in air traffic 
separation procedures, beyond their current level of conforming to clearances, and seeing and avoiding traffic in 
visual conditions. 

Description of the In-Trail Procedure 

The ground rules for ITP are described in the Operational Services and Environment Description (OSED) 
document [2] from the RTCA/EUROCAE Requirements Focus Group (RFG). The ITP is designed to enable 
altitude changes through flight levels that would otherwise be blocked. The procedure requires the crew to use 
information derived on the aircraft to determine if the criteria required for an ITP are met. In actual airborne 
operation, onboard ITP equipment would receive ADS-B data from aircraft within reception range that are 
broadcasting these data, and derive information required for the ITP from these data. The aircraft-derived 
information includes flight identifier (ID), flight level, same direction, ITP distance, and ground speed 
differential (all relative to potentially blocking aircraft). The ITP equipment receives the ADS-B data and, along 
with onboard navigation data, calculates appropriate separation information for these aircraft and portrays this 
information to the crew (note that this information could be portrayed in a number of ways depending on the 
implementation; one specific implementation will be described subsequently). Using this information, the crew 
determines whether to make a flight level change request, and if so, whether to request a standard or ITP flight 
level change. If the desired flight level appears available but potentially blocking aircraft are observed on the 
intervening flight levels and the requested flight level is no more than 4,000 ft from the initial flight level, then 
the crew would evaluate the available information for these potentially blocking aircraft to determine if they can 
be used as reference aircraft in an ITP flight level change request. An aircraft at an intervening flight level can 
be used as a reference aircraft if it meets the following criteria: 
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same direction of flight as the ITP Aircraft, ± 45 degrees, 
qualified ADS-B data are being received from the aircraft, and 
ITP distance/speed criteria are met. 

The ITP equipment calculates for each aircraft the ITP distance, which is the difference in distance to a 
common point as shown in Figure 3 (for identical ground tracks, ITP distance is simply the distance between the 
two aircraft). The ITP speed criterion is a ground speed differential, also calculated by the ITP equipment and is 
simply the difference in ground speed between the two aircraft. A positive ground speed differential is one in 
which the ITP distance between the two aircraft is decreasing. 



Figure 3. ITP distance (non-identical ground tracks vs. identical ground tracks) 


A reference aircraft must meet the following ITP distance/speed criteria: 

ITP distance at least 15 nm, and positive ground speed differential of 20 knots (kts) or less, or 


ITP distance at least 20 nm, and positive ground speed differential of 30 kts or less. 


Up to two reference aircraft can be included in an ITP flight level change request to ATC. The reference 
aircraft are identified in the request by call sign and the ITP distance. If more than two potentially blocking 
aircraft meet the criteria for reference aircraft, then the crew would identify the one or two that, in their 
judgment, would be most likely to block the flight level. Typically this would be the closest one or two such 
aircraft, ahead of and/or behind the ITP aircraft, and on either one or two intervening flight levels, depending on 
the situation. 


All possible ITP geometries are permissible as long as all aircraft in the request meet the criteria for 
reference aircraft. Possible geometries include: 

following climb/descent; 

leading climb/descent; 

combined leading-following climb/descent; 
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combined leading-leading climb/descent (with reference aircraft on different flight levels); and 

combined following-following climb/descent (with reference aircraft on different flight levels). 

The reference aircraft is located behind the ITP aircraft during a leading maneuver, and the reference aircraft is 
located ahead of the ITP aircraft during a following maneuver. 

Given that the ITP is an airborne traffic situation awareness application, no change in the responsibilities of 
either pilots or controllers takes place. The flight crew continues to be responsible for the operation of the 
aircraft and conformance to its clearance, and the controller continues to be responsible for separation and the 
issuance of clearances. However, the controller would use additional information from the ITP aircraft’s request 
to determine if separation can be assured and a clearance can be issued. 

Upon reception of an ITP request, ATC can: 

deny the ITP flight level change request due to traffic or other constraints; 

approve a standard flight level change if sufficient separation exists and ITP clearance is not necessary; 
or 

issue an ITP flight level change clearance, identifying the reference aircraft call sign(s). 

ATC determines if standard separation will be met for all aircraft at the requested flight level and at all flight 
levels between the initial flight level and requested flight level. If so, a standard (non-ITP) flight level change 
clearance may be granted. Otherwise, if the reference aircraft are the only blocking aircraft, the controller 
evaluates the ITP request. ATC determines if the reference aircraft have been cleared to change speed or change 
flight level, or are about to reach a point at which a significant change of track will occur. The controller also 
determines that the requesting aircraft is not referenced in another procedure and that the positive Mach 
difference with the reference aircraft is no greater than 0.04 Mach. If the separation criteria are met at the 
requested flight level with other aircraft, the requesting aircraft itself is not referenced, and the reference aircraft 
are maintaining speed, flight level, and track, then ATC may issue the ITP flight level change clearance. 

If an ITP clearance is received, then the crew must reassess the reference aircraft identified in the clearance 
to assure that the ITP distance/speed criteria are still met before accepting the clearance. If the criteria are no 
longer met, then the clearance must be rejected. 

If the criteria are still met and an ITP clearance has been accepted, the crew should commence the flight level 
change without delay and maintain cruise Mach number and at least 300 feet per minute (fpm) vertical speed 
throughout the flight level change. If this minimum performance cannot be maintained, then regional 
contingency procedures for inability to conform to an ATC clearance should be followed. 

The ITP aircraft crew is not required to monitor the ITP distance to the reference aircraft during the climb or 
descent. The safety of the ITP is assured by the initial conditions, which include the ITP distance, ground speed 
differential, and requested vertical distance for the flight level change, and by the required minimum vertical 
speed during the flight level change. The ITP is completed when the ITP flight crew reports being established at 
the new flight level. 

Current Study 

The EOO HITL ITP Validation Simulation Study was conducted to collect quantitative and qualitative data 
to assess the validity and pilot acceptability of an ITP designed to enable oceanic flight level changes that would 
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not otherwise be possible. The results of this study address fundamental questions regarding the design and 
correctness of the EOO ITP concept of operations. 

Research Objectives and Hypotheses 

The first objective of the EOO HITL ITP Validation Simulation Study was to assess the validity of the ITP 
by determining if subject pilots: a) were able to perform ITP maneuvers during simulated flights over the 
Atlantic Ocean and b) found the procedural steps that they were instructed to use while executing ITP 
maneuvers to be correct, complete, and appropriately specified. 

It was hypothesized that subject pilots would be able to perform ITP maneuvers during simulated oceanic 
flights (i.e., that the instructed procedural steps would be performed correctly and in the appropriate order) and 
that subject pilots would not find any missing, incomplete, or extraneous procedural steps associated with the 
ITP. To test these hypotheses, 12 commercial airline pilots with current oceanic experience were asked to 
perform standard and ITP flight level change maneuvers while using a desktop aircraft simulator to fly along 
assigned routes in the NATOTS. The validity of the ITP was evaluated using the subject pilots’: 1) flight level 
change procedure selection errors; 2) execution errors for selected flight level change procedure; 3) post- 
scenario questionnaire responses; 4) post-experiment questionnaire responses; and 5) feedback obtained during 
post-experiment group debrief sessions. 

The second objective of the EOO HITL ITP Validation Simulation Study was to assess pilot acceptability of 
the ITP. In addition to collecting subject pilots’ impressions of the overall acceptability of the ITP, the study 
attempted to determine if subject pilots found that: a) the level of workload that they experienced while 
performing ITP maneuvers was acceptable and b) the ITP will likely provide perceived benefits to flight crews, 
passengers, and airline operators in the form of improved traffic situation awareness, smoother rides, and fuel 
savings. 

It was hypothesized that subject pilots would find the workload level associated with performing ITP flight 
level changes to be acceptable [i.e., that a subjective workload rating of “3” or less would be chosen using the 
Modified Cooper-Harper (MCH) Rating Scale (a description of this rating scale is provided in the “Dependent 
Measures” section and in Appendix A)] and that the subjective workload level experienced during ITP flight 
level changes would not be significantly higher than that experienced during standard flight level changes. 
Additionally, it was hypothesized that subject pilots would find the ITP to be beneficial in situations where 
climbs or descents would not otherwise be possible. The acceptability of the ITP was evaluated based on the 
subject pilots’: 1) workload ratings; 2) post-scenario questionnaire responses; 3) post-experiment questionnaire 
responses; and 4) feedback obtained during post-experiment group debrief sessions. 

Method 

Participants 

Participants consisted of 12 commercial airline pilots employed by a single major U.S. air carrier. Each pilot 
had completed the ITP Display Interface User Survey (included in Appendix B) and had expressed an interest in 
participating in NASA LaRC’s investigations of the ITP concept. Survey respondents were invited to serve as 
subject pilots in the HITL ITP Simulation Validation Study because it was anticipated that their exposure to the 
ITP concept via the survey would reduce the training time required to prepare them for the performance of the 
study’s evaluation tasks. Survey respondents who were currently flying or who had recent experience flying 
Boeing 747 and/or 777 aircraft were invited to serve as subject pilots since their experience with glass cockpit 
technology would facilitate their training and use of the experiment’s desktop flight simulator. 
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All of the subject pilots were male and ranged in age between 42 - 59 years [mean (M) = 49, standard 
deviation (SD) = 6]. Five of the participants were captains, and the other seven were first officers. On average, 
the participants had 18 years of airline experience (M = 18.6, SD = 7.2, Range =11- 32) and over 9,000 hours 
of airline flying experience (M = 9,892, SD = 5,829, Range = 3,000 - 23,000). At the time of the study, six of 
the participants served as 747-400 pilots; five of the participants served as 777-200 pilots; and 1 participant 
served as a 767-300 pilot; however, the 767 pilot had recent experience flying a 747 aircraft. All subject pilots 
had experience flying in the NATOTS and had, on average, completed nearly 60 oceanic flights during the 
previous year (M = 59.7, SD = 32.5. Range = 25 - 120). When asked to rate their level of familiarity with flying 
oceanic routes on a scale from 0 (“very unfamiliar”) to 10 (“very familiar”), the mean response was 9.38 (SD = 
0.71). 

All of the participants had previous experience with the use of data-link communications, and two of the 
participants had previous experience using EFB devices. All of the subject pilots viewed themselves as being 
relatively computer savvy. When asked to rate their level of computer usage on a scale from 0 (“I never use a 
computer”) to 10 (“I use a computer multiple times every day”), the mean response was 8.75 (SD = 1.71). 

All of the subject pilots participating in the ITP HITL simulation study did so voluntarily and were 
compensated for their participation. Throughout the experiment, all participants were treated in accordance with 
the “Ethical Principles of Psychologists and Code of Conduct”[3]. 

Test Facilities and Apparatus 

The EOO HITL ITP Validation Simulation Study took place in the Air Traffic Operations Laboratory 
(ATOL) at NASA LaRC. The ATOL is a facility that hosts the Airspace and Traffic Operations Simulation 
(ATOS), a distributed simulation environment, built using a High-Level Architecture (HLA) network 
communications model, enabling the investigation of overall airspace system behaviors while retaining highly 
sophisticated models of the various system components. These system components include piloted computer 
workstation-based commercial transport and general aviation aircraft simulations, links to onsite high-fidelity 
motion-based flight deck simulators, background traffic generators, links to offsite radar and ATC host 
simulations, models of various communications and surveillance infrastructure elements (e.g., Flight 
Information Services Broadcast, or FIS-B), a simulation manager that controls modes and timing, and data 
collection and analysis tools [4]. A simplified diagram of the basic architecture of ATOS is shown in Figure 4. 
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Piloted ASTOR \Aforkstations 


Figure 4. Simplified basic architecture of the ATOS system components 

For this experiment, the components of ATOS that were used included the commercial transport aircraft 
simulation [i.e., the Aircraft Simulation for Traffic Operations Research (ASTOR)], a new EFB simulation that 
hosted the Oceanic Operations application, the background traffic generator and ATC simulation (i.e., TMX), 
the simulation manager, and the data collection and analysis tools. The ASTOR, EFB, and TMX components 
will be described in more detail in the following sub-sections. 

Aircraft Simulation for Traffic Operations Research (ASTOR) 

The aircraft simulation used for this experiment, known as the ASTOR, is a medium-fidelity computer 
workstation-based desktop flight simulator whose displays and control panels are representative of a current- 
generation transport aircraft flight deck (e.g., a Boeing B-777). As shown in Figure 5, the pilot interface to 
ASTOR includes an Electronic Flight Instrumentation System Control Panel (EFISCP), auto-flight system Mode 
Control Panel (MCP), Multifunction Control and Display Unit (MCDU), Engine Indication and Crew Alerting 
System (EICAS) display, Display Select Panel (DSP), Primary Flight Display (PFD), Navigation Display (ND), 
Radio Tuning Panel (RTP), Transponder Control Panel (XCP), throttle control stand panel including a variety of 
aircraft configuration controls, and a simulation status panel. 
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Primary Navigation Radio Transponder Throttle Simulation 

Flight Display Tuning Control Control Status 

Display (ND) Panel Panel (XCP) Stand Panel Panel 

(RTP) 

Figure 5. General layout of the on-screen pilot interface to ASTOR 

For this experiment, the EICAS display also doubled as the Multi-Function Display (MFD) that hosted the 
Controller Pilot Data Link Communications (CPDLC) application, which was used by the subject pilots to send 
requests to and receive clearances from the ATC simulation. Although in a real flight deck this CPDLC 
application is normally accessed through a separate MFD located below the EICAS display, space 
considerations required that the MFD and EICAS share a common display area. The subject pilots used the DSP 
to switch between the two displays. 

Each ASTOR included a research prototype Flight Management Computer (FMC) simulation that was 
capable of supporting most normal oceanic flight management operations, such as route planning and 
modification, lateral and vertical trajectory computation, and lateral and vertical guidance generation. The 
subject pilots interacted with the FMC using the MCDU. The aircraft and engine performance models used for 
this experiment were representative of a large twin-engine commercial transport aircraft, and although simulated 
winds aloft were present in each scenario, no wind prediction errors were introduced that would have resulted in 
the aircraft flying a different path than predicted by the simulated FMC. All ASTOR components, including the 
FMC and other elements of the autoflight system, communicated with each other using an onboard avionics 
architecture based on a simulated ARINC 429 digital data bus [5]. 

Surveillance data on those nearby traffic aircraft which were designated as ADS-B equipped aircraft were 
provided to each ASTOR aircraft using simulated ADS-B reports [6]. The modeled ADS-B reports included the 
state vector, mode status, air-referenced, target state, and trajectory change reports, and the contents of these 
reports conformed to the Minimum Aviation System Performance Standards (MASPS) for ADS-B [7]. Range 
limitations and interference effects were modeled to provide realistic representations of the data that would be 
available in the real world from ADS-B equipped aircraft. Surveillance data for aircraft not equipped with 
ADS-B were provided using simulated Traffic Alert and Collision Avoidance System (TCAS) information, 
which was more limited than the ADS-B data and consisted primarily of range, azimuth, and altitude data. 
Symbols for the TCAS data were displayed on the ND, while symbols for both the TCAS and ADS-B data were 
displayed on the simulated EFB. 
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Electronic Flight Bag (EFB) 

In addition to the main ASTOR displays and control panels, which were presented on two 19-inch liquid 
crystal display (LCD) flat panel screens, the pilot interface for this experiment also included an EFB simulation. 
This EFB simulation hosted the ITP application and was run on a separate tablet personal computer (PC) to the 
left of the two LCD screens (see Figure 6). Interaction with the ITP Oceanic Operations application on the 
tablet PC was accomplished by the subject pilots using a separate computer mouse from the one used to interact 
with the main ASTOR displays and control panels. 



Figure 6. Location of simulated EFB relative to the main ASTOR displays 


The ITP application interface was designed for this experiment. The design goal was to minimally satisfy 
the OSED requirements, without adding any feature or display element that was not clearly traceable to a 
specific requirement. In addition, the following design assumptions were made: 1) with the exception of 
selecting the desired flight level, the pilot interface would be totally passive, so that no pilot interaction would 
be required other than viewing the display; 2) the pilot interface would show the results of evaluating all 
relevant procedural criteria associated with each maneuver, so that pilots would not be required to remember 
these criteria themselves; and 3) the pilot interface would require very little information from the onboard 
avionics, limited to position, ground speed, ground track, and barometric altitude. 

Additionally, a larger-scale survey of oceanic airline pilots was conducted as a means to better evaluate the 
current ITP interface display design (Appendix B). Based on the results from the user survey (Appendix C), a 
prototype implementation of the ITP interface was created and integrated with the existing EFB software 
running on the tablet PC as part of ASTOR. An example of the final interface display is shown in Figure 7. The 
details and purpose of each element of the display are explained in the Flight Manual Bulletin (FMB) presented 
in Appendix D, which was given to the subject pilots as part of their experiment training materials. 
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Figure 7. Final ITP application interface display as used in the experiment 

In addition to the ITP application interface, a prototype implementation of the logic contained in the OSED 
document describing the ITP maneuver constraints was also created. This ITP algorithm logic, described in a 
formal specification language [8], was used to determine the status of many of the elements on the ITP interface 
display. For example, the vertical profile view of nearby traffic aircraft on the display only showed those 
aircraft that met the definition of “same track” and “same or opposite direction,” so the ITP algorithm selected 
these aircraft according to the appropriate rules. Likewise, the “range” value shown beneath traffic aircraft at 
intervening altitudes was computed as a distance to or from a common point by the ITP algorithm (i.e., the value 
displayed is the ITP distance for that aircraft). Additionally, the ITP algorithm actually applied the reference 
aircraft criteria to the appropriate traffic aircraft, so that the interface display could clearly label as “NO REF” 
(no reference) those aircraft that failed to meet the criteria and were therefore ineligible to be included as 
reference aircraft in an ITP flight level change request. 

Traffic Manager (TMX) 

A desktop ATM simulation program called the Traffic Manager (TMX) performed the function of 
background traffic generator and ATC simulation. TMX was also extensively used for scenario development 
and data collection. TMX (Figure 8) is a medium- fidelity computer workstation-based desktop simulation 
application designed for interaction studies of aircraft in a future ATM environment. TMX was originally 
developed by the National Aerospace Laboratory (NLR) in The Netherlands, and it serves as a stand-alone 
traffic simulator, flight simulation scenario generator, scenario editor, experiment control station, data-recording 
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tool, and rapid prototyping environment. Both NLR and NASA LaRC have continued to enhance and improve 
TMX, making it a valuable asset to many ATM research projects [9]. 



Figure 8. Traffic Manager user interface 


The simulator is capable of simulating up to 2,000 aircraft simultaneously, and each aircraft uses a six- 
degrees-of-freedom dynamics model, augmented with performance parameters from the EUROCONTROL Base 
of Aircraft Data (BADA) database [10]. Other features are shown in Table 1. 

Table 1. TMX functionality 


Gate to Gate Operations: includes approach and taxi control 

Autoflight Model: basic altitude, heading, and speed modes, plus Flight Management 
System (FMS) modes with autothrottles and Required Time of Arrival functionality 
ADS-B Model: includes range limits 

Airborne Separation Assistance Systems (AS AS): conflict detection, resolution, and 

prevention systems selectable among multiple variants 

Airborne Precision Spacing: for merging and spacing operations 

Pilot Model: includes parameters for reaction time and scheduling of tasks 

Wind Model: three-dimensional “truth” and predicted wind fields 

Weather Model: includes moving weather cells 

Datalogging: time and event based 


TMX also supports external connection interfaces (including HLA) to connect to full motion simulators and to 
integrate with the NASA ATOS. 

For use during the EOO HITL ITP Validation Simulation Study, a new ATC model was developed to 
provide the necessary separation assurance services. The model consisted of individual ATC centers, each of 
which was capable of handling multiple requests simultaneously. The communication between the ATC model 
and the requesting aircraft was accomplished via data-link, and all data-link messages were formatted using a 
consistent set of syntax rules according to the ARINC 702 standard. Controller response times were modeled 


12 




based on actual oceanic data obtained from oceanic air traffic service providers (ATSP), and ranged from one to 
four minutes depending on the operation. 

Pilot Procedure 

During the experiment scenarios, each subject pilot sat at an ASTOR station and was instructed not to 
interact with any of the other experiment participants. The initial condition of each ASTOR simulation was set 
so that the ownship aircraft was positioned roughly one hour into a flight occurring on a NATOTS track, with 
the Flight Management System (FMS) initialized and the auto-pilot fully coupled. Upon start of the simulation, 
the subject pilots were instructed to try to get as close as possible to the recommended flight level as reported on 
the Vertical Navigation (VNAV) page of the MCDU (Figure 9). 



Figure 9. FMS recommended Flight Level 


To determine the appropriate flight level change request, the recommended flight level had to be selected on 
the EFB’s Oceanic Operations application (shown in green in Figure 10). Once selected, the application 
displayed aircraft symbols and data tags for all aircraft at the current flight level, desired flight level, and 
intervening flight levels that were transmitting position information. 
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Figure 10. EFB desired altitude selection 


Using this traffic information, the subject pilot determined whether to request a standard or an ITP flight 
level change. Subject pilots were advised that “if in doubt, request an ITP.” That is, if the subject pilot was 
unsure if standard separation would exist at one or more of the intervening flight levels, then the pilot should 
request an ITP to maneuver through the intervening flight level(s) (assuming all of the ITP criteria are met) 
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rather than request a standard flight level change. While an ITP request would be more involved, it would allow 
the controller to use less restrictive separation standards. Alternatively, if standard separation was clearly 
apparent [e.g., no aircraft were visible on the intervening flight level(s)], then the subject pilot should request a 
standard flight level change. If aircraft on the intervening flight level(s) clearly blocked the desired flight level 
change under even the ITP criteria, then no request should be made. 

If the subject pilot determined that an ITP request was required, he chose the one or two potentially blocking 
aircraft, ahead of and/or behind the ownship aircraft and on either one or two intervening flight levels, that 
would most likely block the flight level change. The ITP criteria would be evaluated for these aircraft to 
determine whether or not they could be used as reference aircraft in an ITP request. 

To request an ITP flight level change, the subject pilot used a data-link interface, similar to that required to 
make a standard flight level change request. Additional ITP-specific information was entered on the available 
free text lines. With this design choice, an actual flight deck implementation would not require modifications to 
the aircraft’s existing data-link interface or underlying software. 

Subject pilots entered free-text on the MCDU scratchpad and transferred the text to the ATC data-link page 
(Figure 11). 
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Figure 11. Constmcting ITP Flight Level change request 

The keyword “ITP” had to start the first (and only the first) free text line and was followed by the information 
pertaining to each of the one or two reference aircraft, according to the following format: 


F/<reference aircraft flight id>/nn 
Where: 


or L/<reference aircraft flight id>/nn 


F/ means that the ITP Aircraft is following this reference aircraft 
L/ means that the ITP Aircraft is leading this reference aircraft 
/nn is the ITP Distance for this reference aircraft, in nm 

After the ITP request was sent, ATC (simulated in software) would evaluate the request and reply either by 
denying the request, approving a standard flight level change, or approving an ITP flight level change. Upon 
reception of an ITP clearance (Figure 12), the subject pilot had to reassess the reference aircraft identified in the 
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clearance to assure that the ITP criteria were still met before accepting the clearance. The clearance had to be 
rejected if the criteria were no longer met. 



Figure 12. Reception of ITP clearance 


Once a clearance was accepted, the subject pilot was instructed to arm an automatic report to inform ATC 
once the reported altitude was established, per standard operating procedures for flight level changes in the 
simulated NATOTS airspace and with this data-link capability. The subject pilot was then able to commence 
the flight level change without delay, while maintaining the current cruise Mach number and at least 300 fpm 
vertical speed throughout the flight level change. The subject pilots were at liberty to choose their own means 
of changing altitude, while adhering to the procedure requirements. No further reference to the Oceanic 
Operations application (e.g., traffic monitoring) was required after the climb or descent was initiated. 


Experiment Design 


The experiment design used for data collection was an 8 x 2 full-factorial, within-subject design (Figure 13). 
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Figure 13. Experiment design matrix 

The eight levels of the Type of Flight Level Change Maneuver independent variable were: 1) Standard Climb, 2) 
Standard Descent, 3) ITP Following Climb, 4) ITP Following Descent, 5) ITP Leading Climb, 6) ITP Leading 
Descent, 7) ITP Combined Climb, and 8) ITP Combined Descent. The two levels of the Achievability of Flight 
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Level Change Maneuver independent variable were: 1) Yes, and 2) No. Twelve subject pilots (Sun) performed 
all 16 test conditions in random order. 

In addition to the experiment’s 16 test conditions, all of which involved a flight level change of 2,000 ft, 
seven additional simulated flight scenarios involving flight level changes greater that 2,000 ft were completed 
during a post-experiment supplemental data collection session. The additional scenarios were designed to get 
feedback on scenarios that were more complex. Due to the fact that these scenarios were not part of the matrix, 
not all subject pilots flew the same additional scenarios. 

Independent Variables 

The two independent variables included in the experiment design were Type of Flight Level Change 
Maneuver and Achievability of the Flight Level Change Maneuver. This second independent variable, 
Achievability of the Flight Level Change Maneuver, will be more fully explained in a subsequent sub-section 
describing the design of the experiment scenarios. 

Additional Variable of Interest 

While not recognized as a factor salient enough to be included in the experiment design as an independent 
variable, Direction of Flight was identified as a variable of interest since the Oceanic Operations application that 
subject pilots used during the experiment provided a vertical profile view of aircraft traveling from left-to-right 
across the lower portion of an EFB display screen. Although subject pilots were expected to be capable of using 
the Oceanic Operations application’s vertical profile view equally well during simulated flights performed East- 
to-West and West-to-East over the Atlantic Ocean, the decision was made to evaluate the effects that Direction 
of Flight might have on the ability of subject pilots to interpret information presented in the vertical profile 
view, particularly when performing simulated Westbound flight scenarios. 

Each subject pilot flew East-to-West during half of the test conditions and flew West-to-East during the 
remaining half of the test conditions. The order in which Eastbound versus Westbound flights were performed 
was completely randomized. However, each simulated flight scenario was flown East-to-West by half of the 
subject pilots and was flown West-to-East by the other half of the subject pilots to ensure that each scenario was 
flown Eastbound and Westbound an equal number of times. 

Dependent Measures 

Selection Errors. Prior to making a flight level change request, subject pilots were required to determine 
which type of flight level change maneuver, if any, was most appropriate given the situation at hand. It was 
necessary to determine whether the most appropriate course of action was to request a standard flight level 
change, request an ITP flight level change, or elect to make no request and remain at the current flight level. 

During four of the experiment’s simulated flight scenarios, subject pilots were expected to request a standard 
flight level change. During the other 12 scenarios, subject pilots were expected to consider requesting an ITP 
flight level change but, ultimately, were expected to refrain from making a flight level change request during 
three of the 12 “ITP scenarios.” These three scenarios were designed so that either: 1) the distance of a 
reference aircraft did not meet the ITP distance criteria, 2) the ground speed differential of a reference aircraft 
did not meet the required ITP speed criteria, or 3) an observable traffic aircraft was blocking the ITP maneuver. 

Selection errors were logged when subject pilots made a flight level change request that was different from 
the expected request for a given scenario. For example, a selection error was logged if subject pilots requested: 
a) a standard flight level change when they were expected to request an ITP flight level change, b) an ITP flight 
level change when they were expected to request a standard flight level change, or c) an ITP or standard flight 
level change when they were expected to have realized that neither an ITP nor standard maneuver was possible. 
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Execution Errors. Execution errors were evaluated after subject pilots chose to request a given type of 
flight level change maneuver, and any execution errors committed were attributed to the type of flight level 
change actually requested (rather than to the expected flight level change for the scenario, in the case of a 
selection error). An execution error was logged if subject pilots failed to: a) correctly communicate required 
information to ATC while making either a standard or an ITP flight level change request, or b) adhere to the 
aircraft performance criteria required during ITP flight level change maneuvers. Depending on the potential 
consequences associated with a given execution error, it may have been identified as a “safety-related execution 
error.” Criteria for identifying safety-related execution errors are described subsequently in the Results and 
Discussion section. 

Subjective Assessments of the ITP’s Validity. Subject pilots’ perceptions regarding the validity of the ITP 
were collected using a post-scenario questionnaire (Appendix E) that was administered after the completion of 
each test condition and a post-experiment questionnaire (Appendix F) that was administered after the 
completion of the final test condition. Subject pilots used the post-scenario questionnaire to record their 
impressions of the correctness, completeness, appropriate specification, and logical sequencing of the procedural 
steps that they were instructed to use while executing ITP maneuvers. The post-experiment questionnaire was 
used by the subject pilots to document any performance concerns they had with the ITP (e.g., the ability to 
maintain climb rate and/or Mach). Additional comments regarding the validity of the ITP were also collected 
from the subject pilots during the post-experiment group debrief session. 

Subjective Assessments of the ITP’s Acceptability. Subject pilots’ perceptions regarding the acceptability 
of the ITP were collected using the post-scenario questionnaire and the post-experiment questionnaire. The 
post-scenario questionnaire provided subject pilots with a means through which to document any alternative 
ways that they would have preferred to perform a given flight level change maneuver. Examples of the 
information recorded by the post-experiment questionnaire include subject pilots’ impressions regarding: the 
level of safety associated with performing the ITP as compared with current day procedures; how the workload 
required to perform standard flight level changes during the experiment compared with the workload required to 
perform ITP flight level changes; and the perceived benefits that the ITP might have for flight crews, 
passengers, and airline companies. Subject pilots were also encouraged to comment on their perceptions of the 
ITP’s acceptability during the post-experiment group debrief session. 

Subjective Assessments of the ITP’s Workload. Subjective assessments of the workload associated with 
performing the experiment’s simulated flight scenarios were obtained through the use of the Modified Cooper- 
Harper (MCH) Rating Scale [11] (Appendix A), the post-experiment questionnaire, and the post-experiment 
group debrief sessions. The MCH scale required subject pilots to make a series of decisions regarding: whether 
or not the instructed task could be accomplished most of the time; if adequate performance was attainable (i.e., 
if errors were small and inconsequential); and whether or not the level of mental workload required by the 
instructed task was acceptable. Upon answering questions according to a predetermined logical sequence, an 
overall rating ranging from “1” (indicating that the instructed task was very easy/highly desirable; operator 
mental effort was minimal; and desired performance was easily attainable) to “10” (indicating that the instructed 
task was impossible; it could not be accomplished reliably) was selected (Figure 14). 
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Figure 14. Modified Cooper-Harper (MCH) Rating Scale 


The post-experiment questionnaire was used to record the subject pilots’ descriptions of how the workload 
required to perform standard flight level changes during the experiment compared with the workload required to 
perform the ITP flight level changes. Feedback regarding their impressions of the workload associated with 
performing simulated flights East-to-West versus West-to-East over the Atlantic Ocean was obtained from the 
subject pilots during the post-experiment group debrief sessions. 

Scenario Design 

The experiment’s simulated flight scenarios were designed in accordance with the requirements of the 8 x 2 
experiment design matrix. The 16 experiment matrix scenarios involved maneuvers through one intervening 
flight level, while the additional seven supplemental scenarios were designed to evaluate more complex 
situations involving maneuvers through multiple intervening flight levels. 

Every experiment scenario was designed based on the eight types of flight level change maneuvers: 

SC = Standard climb 
SD = Standard descent 
FC = ITP Following climb 
FD = ITP Following descent 
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LC = ITP Leading climb 
LD = ITP Leading descent 
CC = ITP Combined climb 
CD = ITP Combined descent 


and the achievability of the flight level change: 


Yes (AC = Accept) 

No (RJ = Reject, or UA = Unable) 

A flight level change was not achievable if either the target and/or intermediate flight level was blocked by 
traffic aircraft of which the subject pilot was unaware (RJ, i.e., controller rejects the flight level change request) 
or blocked by traffic aircraft of which the subject pilot was aware (UA, i.e., the pilot should not request the 
flight level change, or should reject the ITP clearance received). The achievability of the flight level change 
could be restricted by: 

Distance to reference aircraft not meeting ITP distance criteria (UAD) 

Ground speed differential of reference aircraft not meeting ITP speed criteria (UAG) 


Failed reassessment (i.e., reference aircraft no longer meet ITP criteria upon receiving an ITP clearance, 
so the clearance must be rejected) (UAR) 

Observable traffic blocking the maneuver (UAT) 


To prevent sequence effects, which scenarios would be accepted, rejected, or unable to be completed were 
randomly chosen. This resulted in the following list of scenarios: 


1. SCI AC 

2. SC1RJ 

3. SD1AC 

4. SD1RJ 

5. FC1AC 

6. FC1RJ 

7. FD1AC 

8. FD1UAD 

9. LC1AC 

10. LC1UAR 

11. LD1AC 

12. LD1UAT 

13. CC1AC 

14. CC1UAR 

15. CD1AC 

16. CD1 UAG 


Standard Climb through 1 FL - Accepted 

Standard Climb through 1 FL - Rejected 

Standard Descent through 1 FL - Accepted 

Standard Descent through 1 FL - Rejected 

Following Climb through 1 FL - Accepted 

Following Climb through 1 FL - Rejected 

Following Descent through 1 FL - Accepted 

Following Descent through 1 FL - Unable due to Distance 

Leading Climb through 1 FL - Accepted 

Leading Climb through 1 FL - Unable due to Reassessment of ITP criteria 
Leading Descent through 1 FL - Accepted 
Leading Descent through 1 FL - Unable due to Traffic 
Combined Climb through 1 FL - Accepted 

Combined Climb through 1 FL - Unable due to Reassessment of ITP criteria 

Combined Descent through 1 FL - Accepted 

Combined Descent through 1 FL - Unable due to Ground Speed 


Each scenario was designed using a graphics program. Aircraft significant to the scenario were placed in a 
way that resulted in the desired maneuver, while other aircraft were added to improve the operational realism 
(Figure 15). A full list of all experiment scenarios can be found in Appendix G. Care was taken to prevent 
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unexpected pilot actions by making the desired maneuvers unambiguous. In the example of Figure 15, the blue 
ITP Aircraft was placed such that an ITP request was required, using the red aircraft as a reference. The aircraft 
at the desired altitude was placed outside the oceanic separation requirement of 10 minutes (approximately 80 
nm). 


Owriship: FL 330; 0.31M 
Desired Alt: FL350 



Once created, unique scenarios were combined into experiment sets to present each subject pilot with a 
different maneuver. Experiment sets included all available tracks and were randomly chosen to be either 
Eastbound or Westbound. An example of an Eastbound experiment set is provided in Table 2. 

Table 2. Eastbound experiment set 1 (ITPExpSetlEAST) 


Subject Pilot: 

Track: 

Scenario 

ASTOR 1 

X 

CD1 UAG EAST 

ASTOR 2 

w 

CC1ACEAST 

ASTOR 3 

u 

SCI RJ EAST 

ASTOR 4 

Y 

LD1ACEAST 

ASTOR 5 

T 

FC1 RJ EAST 

ASTOR 6 

V 

SCI AC EAST 


In total, 32 experiment sets were created (16 experiment scenarios x 2 directions of flight). Subject Pilots 1-6 
flew experiment set 1-16, and Subject Pilots 7-12 flew experiment sets 17-32. The scenarios within each set 
were again randomized to prevent any sequence effects during the experiment. 

In addition to the test condition scenarios, the subject pilots were also asked to complete seven supplemental 
scenarios. One of the supplemental scenarios is shown in Figure 16, and the remaining six are presented in 
Appendix H. The scenario shown in Figure 16 involved a 4,000 ft climb by the ownship aircraft (shown in blue) 
with opposing traffic and reference aircraft (shown in red) on different altitudes. 
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Ownship: FL 330; 0.81M 
Desired Alt: FL370 



The purpose of the supplemental scenarios was to test the use of the ITP during more complex situations 
such as flight level changes greater than 2,000 ft, opposite direction traffic, and reference aircraft on more than 
one intervening flight level. All of the supplemental scenarios are listed below. 


1. CC3POSTEXP 

2. CC3LFAC 

3. CC2LFAC 

4. CC2FFAC 

5. CC2FFAC 

6. FC3UAT 

7. CC3 FF UAD 


Combined Climb through 3 FF - heading / Following - Accepted 
Combined Climb through 3 FF - heading / Following - Accepted 
Combined Climb through 2 FF - heading / Following - Accepted 
Combined Climb through 2 FF - Following / Following - Accepted 
Combined Climb through 3 FF - heading / heading - Accepted 
Following Climb through 3 FF - Unable Traffic 

Combined Climb through 2 FF - heading / Following - Unable Distance 


Experiment Procedure 


Subject pilots participated in the EOO HITF ITP Validation Simulation Study in groups of up to four, with 
each group participating in the study over the course of two consecutive days. Subject pilots arrived at the 
ATOF at 8:00 a.m. (EDT) on the first day of the experiment and completed a pre-experiment session, a 
comprehensive period of training exercises (described below in the “Training” section), and eight of the 
experiment’s 16 test conditions before concluding the day’s activities no later than 5 p.m. On the second day of 
the experiment, subject pilots arrived at the ATOF at 7:30 a.m. and completed a brief refresher training session, 
the remaining eight test conditions, a post-experiment questionnaire, a post-experiment supplemental data 
collection session, and a post-experiment group debrief session before the study was concluded at 3 p.m. (Note 
that the subject pilots were provided with several breaks and a 45-minute lunch period each day.) 


The first day of the experiment began with a 45 -minute pre-experiment session during which the subject 
pilots were provided with an introductory overview of NASA’s EOO project. Following this brief presentation 
to the group, the subject pilots were asked to individually read and sign an informed consent document 
(Appendix I) and complete a demographic information questionnaire (Appendix J). After the pre-experiment 
session, subject pilots were given approximately 30 minutes to re-review the Oceanic Operations ITP FMB that 
they had been given the preceding evening and were then given two 45-minute viewgraph-style presentations 
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described in the Training section. Following a 45-minute lunch break, subject pilots completed a “video wall” 
ITP training session. After the video wall training, the subject pilots spent approximately 30 minutes completing 
the hands-on ASTOR training and becoming familiar with the MCH Rating Scale and post-scenario 
questionnaire. Following this, subject pilots were given the opportunity to pose questions to the researchers and 
were then asked to individually complete a post-training quiz (Appendix K). The completion of the quiz and 
final question-and-answer session required approximately 30 minutes. 

During the remaining three hours of the experiment’s first day, the subject pilots used the ASTOR stations to 
individually perform eight of the experiment’s 16 test conditions. After each simulated flight scenario, the 
subject pilots individually completed a MCH Rating Scale response form and a post-scenario questionnaire. 

The second day of the experiment began with a 15 -minute review during which the researchers stepped 
through the performance of an ITP flight scenario to provide the group of subject pilots with a brief period of 
refresher training. Following this, the subject pilots spent approximately three hours using the ASTOR stations 
to individually perform the experiment’s final eight test conditions. As before, the subject pilots individually 
completed a MCH Rating Scale response form and a post-scenario questionnaire after each simulated flight 
scenario. Once the final test condition’s post-scenario questionnaire was completed, each subject pilot was 
asked to complete a post-experiment questionnaire. Completion of the post-experiment questionnaire required 
approximately 30 minutes. 

Following a 45-minute lunch break, the subject pilots used the ASTOR stations to individually complete six 
simulated flight scenarios as part of a 1-hour supplemental data collection effort. Again, each subject pilot 
completed MCH Rating Scale response forms and post-scenario questionnaires after the performance of each 
scenario. Once the supplemental data collection effort was completed, the subject pilots and researchers spent 
the remainder of the afternoon (approximately 1.5 hours) discussing various topics, both planned and 
spontaneous, during a post-experiment group debrief session. 

Training 

The overall objective of the experiment training was to train the subject pilots to performance standard and as 
a result minimize “learning effects” during the experiment scenarios; that is, to reasonably ensure that, prior to 
beginning their first experiment scenario, all participants were uniformly and fully trained in, and familiar with, 
the experiment tasks they were expected to accomplish. To this end, a variety of techniques were employed to 
train the experiment participants, not only to provide redundant coverage of the training material but also to 
accommodate the different learning styles and preferences of different individuals. These techniques are 
described below. 

The experiment training actually began the evening prior to the experiment, when the participants arrived in 
the local area. Upon check-in to their hotel, each participant was provided with a Flight Manual Bulletin (FMB) 
(Appendix D) and instructed to read the bulletin prior to arriving at the experiment site on the following 
morning. This FMB is designed to be in a format similar to an FMB that line pilots might receive from their 
respective companies, and covers all aspects of the ITP concept and specific experiment tasks. The FMB starts 
by describing the background and purpose of the ITP and then describes the ITP systems and operations at a 
conceptual level. It subsequently describes specific controls, indicators, and operational procedures for the 
ASTOR (including the ATC communications data-link interface), EFB, and Oceanic Operations application. 
The FMB then provides a detailed, step-by-step ITP example, from initiation through completion of the flight 
level change, and concludes with an ITP checklist for use by the pilot. 

The training continued on the following morning during the experiment session, when the subject pilots were 
given an additional block of time (approximately 30 minutes) to read and/or review the FMB. This additional 
time block ensured that all subject pilots had seen the FMB prior to further training, even if they were unable or 
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unwilling to read it on the previous evening, and provided additional exposure to the concepts and procedures 
using written communications means. 

After their FMB review time, the experiment subject pilots received two 45-minute viewgraph-style 
presentations, which covered the same material as that contained in the FMB, but in an interactive, classroom 
setting. The first presentation described the ITP background, purpose, use, and procedures from a conceptual 
perspective, and the second presentation illustrated the specific steps necessary to perform experiment tasks 
using the ASTOR. At the outset of these presentations, the subject pilots were informed of the training objective 
and told to expect to see redundant coverage of the concepts and tasks to ensure complete training coverage. 
They were also told that they would be quizzed on the material at the end of the training sessions. Additionally, 
they were told that while questions were encouraged, some answers might have to be deferred until after the 
experiment in the interest of consistent training across participants and sessions, if the question was beyond the 
scope of the planned training. 

Following the classroom presentations, the subject pilots were given “video wall” ITP training. The video 
wall is a large bank of multiple interlinked video screens in the ATOL that allows a complete set of interactive 
simulator displays to be concurrently projected onto a large wall-sized display surface (approximately 6 ft high 
by 9 ft wide), viewable in a classroom presentation setting. The video wall training enabled the subject pilots to 
observe the steps necessary to perform the experiment tasks on the same simulator displays that they would be 
using during the experiment scenarios. These steps were performed by a NASA experiment trainer flying the 
simulator shown on the video wall, using a prepared script to demonstrate: 1) basic ASTOR displays, indicators, 
and operations, 2) the controls, features, and use of the EFB and Oceanic Operations application, and 3) the 
steps required to evaluate, request, and execute both standard and ITP flight level changes. The decision 
processes required for all steps were discussed, and questions for clarification on any aspects of performing the 
experiment tasks were encouraged. 

After the video wall training, the subject pilots were each assigned to an ASTOR station and given the 
opportunity to complete three training flight scenarios that were representative of the experiment scenarios. One 
of the training flight scenarios involved a standard flight level change request, and the other two involved ITP 
flight level change requests. The subject pilots were supplied with the ITP checklist (Appendix L) and were 
coached as necessary through all steps of the training flight scenarios. At the end of each training flight 
scenario, the subject pilots were given the post-scenario questionnaire and MCH Rating Scale to familiarize 
them with the process of filling out these materials prior to commencing the experiment scenarios. When the 
training flight scenarios were finished, the subject pilots were given a written quiz (Appendix K). After the quiz 
was completed, they were presented with the answers and asked to self-grade and correct their quizzes. The 
experiment training was then completed by giving the subject pilots a final opportunity to ask any questions 
about the experiment tasks. 

It should be noted that in addition to the ITP and simulator instruction, the experiment training also included 
instructions that were specific to this experiment protocol. For example, in actual operations there could be 
many factors that might cause a crew to request a flight level change, but for this experiment, the subject pilots 
were instructed to always try to get as close as possible to the recommended flight level as reported by the 
ASTOR FMS. They were also told to expect this guidance to result in an approximately equal number of climb 
and descent requests across all of the scenarios, even though it was noted that in actual operations, climb 
requests would be more common. They were also given general guidance as to when to request a standard flight 
level change versus an ITP flight level change, or when to refrain from making a request. In actual operations, 
this decision would likely depend on a given crew’s judgment, but for this experiment they were advised, as 
previously described in the Pilot Procedure section, that “if in doubt [about adequate separation], request an 
ITP.” 
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Qualitative Data Collection 

A post-scenario questionnaire, a post-experiment questionnaire, and a post-experiment group debrief session 
were used to collect qualitative data from the subject pilots during and after the HITL ITP validation study. 
Each of these qualitative data collection tools is described below. 

Post-Scenario Questionnaire 

Subject pilots completed a post-scenario questionnaire (Appendix E) after each of the experiment’s 16 test 
conditions. The purpose of this questionnaire was to collect data regarding: why subject pilots chose not to 
request a flight level change during certain test conditions (e.g., an observable traffic aircraft prevented the 
execution of an ITP maneuver); the ability of subject pilots to understand why a given flight level change 
request was denied by ATC based on information provided via the ITP display (e.g., a traffic aircraft that was 
not visible to the subject pilot via ADS-B or TCAS prevented the execution of an ITP maneuver); the perceived 
correctness, completeness, appropriate specification, and logical sequencing of the procedural steps outlined for 
use during each ITP flight level change; subject pilots’ suggestions for performing a given flight level change 
differently; and subject pilots’ suggestions for improving the ITP display interface. 

Post-Experiment Questionnaire 

Subject pilots completed a post-experiment questionnaire (Appendix F) after performing the experiment’s 
final test condition. Subject pilots used this questionnaire to record their impressions of: the overall manner in 
which the experiment was conducted; the adequacy of the training that they received prior to performing the 
experiment’s first test condition; the experiment’s simulated flight scenarios and desktop flight simulator; the 
acceptability of the ITP and any ITP performance concerns; their use of the Oceanic Operations ITP Checklist; 
the workload levels that they experienced during the experiment; the perceived benefits of the ITP; the level of 
safety associated with performing the ITP as compared with current day procedures; the ITP display interface 
and the phraseology used when performing the ITP; and the ease of reverting back to standard procedures in the 
event that an ITP flight level change maneuver was abandoned. 

Post-Experiment Group Debrief Session 

During the final 1.5-hour period that they spent together, the subject pilots and researchers engaged in a post- 
experiment group debrief session in which they discussed various topics, both planned (Appendix M) and 
spontaneous, related to the ITP concept, the performance of ITP flight level change maneuvers, and the 
execution of the ITP HITL simulation study. The purpose of this debrief session was to answer any questions 
that the subject pilots wanted to ask and to collect the subject pilots’ comments regarding the experiment itself 
and, more importantly, the validity and acceptability of the ITP. Specific feedback was elicited from the subject 
pilots regarding their impressions of: the benefits and operational improvements potentially gained through the 
use of the ITP; possible ITP display enhancements that could improve performance and situation awareness; the 
usefulness of the ITP display in domestic en-route or terminal environments; any ITP safety concerns; and how 
pilots might react during a variety of situations that could be encountered while performing the ITP [e.g., what 
kind of action(s) pilots might take if they observed that their separation from a reference aircraft had decreased 
to a certain distance during an ITP maneuver]. 

Quantitative Data Collection 

Quantitative data were collected using TMX. The data that were of most interest included the correctness of 
the request and the adherence to the procedural requirements. For that purpose TMX recorded the data-link 
requests, ATC assessments and procedure adherence. Furthermore, track data (comma-delimited aircraft 
parameters data) were collected to analyze aircraft performance and state. 
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Every request received by “ATC” was validated against an assessment that TMX made based on the data 
contained in its internal ADS-B and ITP models. Disparities between the subject pilot request and the request 
assessment by TMX were identified and logged as an assessment failure (i.e., selection error). In addition to the 
request assessment and validation (i.e., execution errors), TMX also logged the number of times that aircraft 
failed to maintain assigned Mach number and the number of aircraft failing to maintain a minimum vertical 
speed of 300 fpm during the maneuver. 

All data-link messages between the subject pilots and the ATC model were logged. This included position 
report messages as well as all flight level change request related messages. Each message was logged with a 
timestamp (both simulation wall-clock time as well as simulation run-time), source identifier, and destination 
identifier. These data could be used to examine whether or not the subject pilot followed all correct procedural 
steps. 

TMX also logged specific aircraft parameters to track the position and state of all aircraft in the scenario. 
These track or history data were logged in a comma-delimited file so that a spreadsheet program could be used 
for data analysis. 

In case any ambiguity remained, either about the request or the procedural steps taken, it was possible to 
playback and review the ASTOR displays, showing exactly what happened during the experiment run. This 
review provided the final determination as to whether or not the subject pilot followed the procedure correctly. 

Results and Discussion 

Overview 

In presenting and discussing the results obtained from the experiment, it is useful to start by revisiting the 
initial two objectives of the study: to assess 1) the validity of, and 2) pilot acceptability of, the ITP. This 
experiment represented a first opportunity to present line pilots with the ITP in simulated operations, with an 
initial prototype Oceanic Operations application, and to evaluate their ITP performance and impressions. It was 
not intended as a quantitative safety assessment of the ITP nor of the initial prototype application, but rather as 
an initial qualitative assessment, and therefore no specific success/failure criteria were defined a priori to 
evaluate ITP validity. Similarly, the only acceptability criterion established beforehand was that workload 
ratings of “3” or less on the MCH Rating Scale would be considered acceptable. 

When the overall results are examined, it is apparent that in the majority of scenarios, subject pilots were 
able to correctly assess the traffic situation and select an appropriate response (i.e., either a standard flight level 
change request, an ITP request, or no request), and to execute their selected flight level change procedure, if 
any, without error. It is also apparent that the workload ratings for ITP maneuvers were operationally 
acceptable and not substantially higher than for standard flight level change maneuvers and that, for the majority 
of scenarios and subject pilots, the subjective acceptability ratings and comments for ITP were generally high 
and positive. It could then be asserted that, from a first-order qualitative point of view, the ITP is generally both 
valid and acceptable. However, the error rates for ITP maneuvers were higher than for standard flight level 
changes, and these errors have design implications for both the ITP and the prototype traffic display. 

The next two sub-sections present quantitative data for, and statistical analyses of, the subject pilots’ 
procedure selection errors and execution errors, respectively, along with discussions of their implications. 
Subsequent sub-sections present results of the pilots’ subjective assessments of the ITP’s validity, acceptability, 
and workload, respectively, along with statistical analyses of the workload results. The final sub-section presents 
results from the additional, more complex ITP scenarios that the pilots performed as part of the supplemental 
data collection effort after the experiment scenarios. 
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A 5 -percent significance level for the statistical analyses of all data collected in this experiment was set a 
priori. 

Selection Errors 

Each scenario presented to the subject pilots was designed to elicit one expected action from the pilot. That 
is, for each scenario, the subject pilot was expected to make either a standard flight level change request, or an 
ITP request of some type (leading, following or combined), or no request, based on their evaluation of the traffic 
situation displayed on the Oceanic Operations application. For example, “standard” scenarios were designed 
with no traffic displayed on the intervening flight level, so there was no need for an ITP request and a standard 
flight level change request would be appropriate. Similarly, “ITP” scenarios were designed with traffic 
displayed on the intervening flight level that would block a standard request, based on NATOTS separation 
standards. If the blocking aircraft met ITP reference aircraft conditions, then the subject pilot was expected to 
make an appropriate ITP flight level change request; otherwise, the subject pilot was expected to make no 
request. During their training, the subject pilots were instructed to consider an ITP request, rather than a 
standard request, for any scenario in which they were unsure whether NATOTS standard separation existed at 
the intervening flight level. 

It should be noted that in today’s flight environment it would not be erroneous for a pilot to request a flight 
level change in the presence of blocking traffic; in contrast with the air traffic controller’s responsibilities, the 
pilot is currently not expected to be aware of all other conflicting traffic nor of the specific separation rules in 
effect, and can make a request at any time. This would also be the case in a future ITP environment; a correctly- 
made standard or ITP request would not be considered erroneous if made in the presence of blocking traffic (but 
the controller would deny the request due to that traffic, just as in today’s flight environment). In this 
experiment, though, the subject pilots were asked to use the traffic information on the Oceanic Operations 
application along with their training guidelines and best judgment in selecting an appropriate action, so 
unexpected actions (including ITP requests with missing or extra reference aircraft) were considered “selection 
errors .” In effect, the selection error measurement is an assessment of the subject pilots’ understanding of, and 
judgment in, the use of the displayed traffic information to make an informed request to ATC. 

Collectively, the 12 subject pilots performed 192 simulated flight scenarios while completing the 
experiment’s 16 test conditions: 

4 scenarios during which subject pilots were expected to request standard flight level changes x 12 
subject pilots = 48, and 

12 scenarios during which subject pilots were expected to consider requesting ITP flight level changes x 
12 subject pilots= 144. 

From this point forward, the scenarios during which subject pilots were expected to request standard flight 
level changes will be referred to as “expected standard scenarios,” and the scenarios during which subject pilots 
were expected to consider requesting ITP flight level changes will be referred to as “expected ITP scenarios.” 
Note that in 3 of the 12 (or 36 of the 144 total) expected ITP scenarios, the pilots were ultimately expected to 
make no request after evaluating the scenarios, due to either observable blocking aircraft or ITP criteria not 
being met. 

Overall, the subject pilots made 19 selection errors out of the 192 scenarios, yielding an overall selection 
error rate of 9.9%. Of these 19 selection errors, only one involved inappropriately requesting a standard flight 
level change (when no request was expected, due to observable blocking traffic), but 18 involved requesting an 
inappropriate ITP flight level change (six when a standard flight level change was expected, eight when a 
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different ITP request geometry was expected, and four when no request was expected due to observable 
blocking traffic). 

During the 48 expected standard scenarios, subject pilots made a total of six ITP flight level change requests. 
That is, 12.5% of the time subject pilots requested an ITP flight level change when they were expected to 
request a standard flight level change. 

Thirteen selection errors occurred during the 144 expected ITP scenarios resulting in an ITP selection error 
rate of 9.0%. Four of the 13 ITP selection errors involved instances in which subject pilots requested an ITP 
flight level change when they were expected to have realized that an ITP maneuver was not possible because an 
observable traffic aircraft was blocking the ITP maneuver. Seven ITP selection errors occurred when subject 
pilots included an additional reference aircraft (but located at the desired, rather than intervening, flight level) in 
their request for an expected leading or following ITP scenario. While allowed by the ATC module, the 
inclusion of an additional reference aircraft in these flight level change requests resulted in requests being made 
for combined ITP flight level change maneuvers instead of for the expected leading or following ITP flight level 
change maneuvers. Although this type of ITP selection error may be attributable to subject pilots’ attempts to 
assist ATC by providing additional information, a reference aircraft (by definition) cannot be located at the 
desired flight level, and subject pilots were trained repeatedly with respect to this definition (these errors will be 
discussed in more detail later in this section). The remaining ITP selection error involved a subject pilot 
requesting a standard flight level change when he was expected to have realized that a standard flight level 
change was not possible due to blocking aircraft, nor was an ITP maneuver possible because the ground speed 
differential of the blocking aircraft did not meet the required ITP speed criteria for a reference aircraft. 

The error rates associated with subject pilots’ expected standard and ITP flight level change selections are 
shown in Table 3. 

Table 3. Percentages of Selection Errors Associated with Expected Standard and ITP Flight Level Change Procedures 


Type of Expected 

Number of 

Number of 

Percentage of Incorrect 

Flight Level 

Selection Errors 

Matrix Scenarios 

Flight Level Change 

Change Procedure 



Procedure Selections 




(Selection Errors) 

Standard 

6 

48 

12.5 

ITP 

13 

144 

9.0 


A McNemar Test (a nonparametric within-subject test appropriate for analyzing two related samples of nominal 
data) [12] was performed to determine if a significant difference existed between the number of selection errors 
that occurred during the 48 expected standard scenarios and the number of selection errors that occurred during 
the 144 expected ITP scenarios. This test revealed no significant difference (p = 0.4244) between standard and 
ITP scenarios for the selection errors measure. 

The overall procedure selection error rate of 9.9% indicates that while in most cases the subject pilots 
correctly understood how to use their traffic display to make an informed request to ATC, there was still some 
confusion. Whether an “error” rate of this magnitude would be operationally acceptable is beyond the scope of 
this study, but in all likelihood this error rate would diminish in actual operations as crews became more familiar 
with air traffic separation standards and the use of a traffic display to determine an appropriate request to make, 
or not make, to ATC. 

It is significant to note that 18 of these 19 selection errors involved inappropriate ITP requests, while only 
one involved an inappropriate standard request. Recall that of the 1 8 selection errors that involved requesting an 
inappropriate ITP flight level change, six occurred when a standard flight level change was expected, and four 
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occurred when no request was expected due to observable blocking traffic (the remaining eight occurred when a 
different ITP request geometry was expected). One explanation for these 10 errors might be that the subject 
pilots were advised in their training that “if in doubt [if standard separation existed], request an ITP.” This 
training, combined with the fact that the subject pilots knew they were participating in an “ITP experiment,” 
might have predisposed them to ask for an ITP even in the presence of blocking traffic, rather than “correctly” 
doing nothing (i.e., making no request) during such a scenario. It might also have predisposed them to request 
an ITP in scenarios where a standard flight level change request was expected because there was no observable 
blocking traffic. However, all six of the standard flight level change scenarios with inappropriate ITP requests 
and all eight of the inappropriate ITP geometry requests had execution errors, implying an incomplete 
understanding of the ITP concepts. These execution errors will be presented and discussed in the following sub- 
section. 

Execution Errors 

Execution errors were evaluated after subject pilots chose to request a given type of flight level change 
maneuver. These errors could occur during the communication of a request (e.g., in the form of typographical 
or procedural errors) or during the performance of an ATC-approved flight level change maneuver. As a result 
of the subject pilots’ selection errors, there was a change in the total number of standard scenarios and ITP 
scenarios during which execution errors could occur. Of the original 48 expected standard scenarios, six were 
“converted into” ITP scenarios by the subject pilots based on their flight level change requests. Since 12 of the 
13 selection errors that occurred during the expected ITP scenarios resulted from incorrect ITP flight level 
change requests, these 12 errors did not affect the number of expected standard scenarios versus expected ITP 
scenarios. However, the remaining ITP selection error resulted in the conversion of an expected ITP scenario 
into a standard scenario. Therefore (and as shown in Table 4): 

48 expected standard scenarios - 6 standard selection errors + 1 ITP selection error = 43 “requested 
standard scenarios,” and 

144 expected ITP scenarios + 6 standard selection errors - 1 ITP selection error =149 “requested ITP 
scenarios.” 


Table 4 shows the breakdown of standard and ITP scenarios requested by each subject pilot. 


Table 4. Expected Standard and ITP Scenarios versus Requested Standard and ITP Scenarios 



Standard 

ITP 

Subject Pilot 

Expected 

Requested 

Expected 

Reqested 

1 

4 

2 

12 

14 

2 

4 

4 

12 

12 

3 

4 

3 

12 

13 

4 

4 

3 

12 

13 

5 

4 

4 

12 

12 

6 

4 

4 

12 

12 

7 

4 

4 

12 

12 

8 

4 

4 

12 

12 

9 

4 

4 

12 

12 

10 

4 

4 

12 

12 

11 

4 

4 

12 

12 

12 

4 

3 

12 

13 

Total Number of 
Scenarios: 

48 

43 

144 

149 
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Subject Pilot #6 is shown to have requested four standard scenarios and 12 ITP scenarios because he committed 
both a standard selection error and an ITP selection error that resulted in expected scenarios being converted 
from one type into another. Therefore, these two selection errors resulted in the final number of requested 
standard and ITP scenarios remaining unchanged for this particular subject pilot. 

Since some execution errors could potentially be associated with more serious consequences than others, 
some of the execution errors described below were identified as being “safety-related execution errors.” In 
general, two types of errors were classified as being safety-related: 1) failures to adhere to performance 
requirements and 2) the inclusion of inaccurate information in a flight level change request. The errors related 
to performance requirements are not unique to the ITP since these errors could occur during any operation but 
might be more serious during an ITP due to reduced separation between aircraft compared to standard non-radar 
flight level changes. However, the errors involving the inclusion of inaccurate information in a flight level 
change request are unique to the ITP in that certain information provided in a flight level change request has the 
potential of either not being verified by ATC or not being seen by ATC as containing inaccuracies. For example, 
a pilot might mistype a reference aircraft’s call sign while requesting an ITP flight level change; but, if this 
mistyped call sign belongs to another aircraft located nearby, this inaccurate information might not be identified 
as such by ATC. While the occurrence of such situations might be rare during actual flight operations, the 
seriousness of the potential consequences associated with such errors warrants their classification as safety- 
related execution errors. 

Subject pilots did not commit any execution errors during the 43 requested standard scenarios. During 29 of 
the 149 requested ITP scenarios (19.5%), subject pilots committed at least one execution error. Of the 29 
requested ITP scenarios involving at least one execution error, nine (6.0%) were identified as involving safety- 
related execution errors. Table 5 depicts the subject pilots’ execution error rates for requested standard and ITP 
scenarios, and Table 6 depicts the subject pilots’ safety-related execution error rates for requested standard and 
ITP scenarios. 

Table 5. Percentages of Requested Standard and ITP Scenarios Involving At Least One Execution Error 


Type of Requested 

Number of 

Number of 

Percentage of Scenarios 

Flight Level 

Scenarios with 

Requested 

with At Least One 

Change Procedure 

At Least One 
Execution Error 

Scenarios 

Execution Error 

Standard 

0 

43 

0% 

ITP 

29 

149 

19.5% 


A McNemar Test was performed to determine if a significant difference existed between the number of 
execution errors that occurred during the 43 requested standard scenarios and the number of execution errors 
that occurred during the 149 requested ITP scenarios. This test revealed that significantly fewer execution 
errors occurred during the requested standard scenarios than during the requested ITP scenarios (p < 0.0001). 

Table 6. Percentages of Requested Standard and ITP Scenarios Involving At Least One Safety-Related Execution Error 


Type of Requested 

Number of 

Number of 

Percentage of Scenarios 

Flight Level 

Scenarios with 

Requested 

with At Least One 

Change Procedure 

At Least One 
Safety-Related 
Execution Error 

Scenarios 

Safety-Related 
Execution Error 

Standard 

0 

43 

0% 

ITP 

9 

149 

6.0% 
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A McNemar Test was also performed to determine if a significant difference existed between the number of 
safety-related execution errors that occurred during the 43 requested standard scenarios and the number of 
safety-related execution errors that occurred during the 149 requested ITP scenarios. This implies that 
significantly fewer safety-related execution errors occurred during the requested standard scenarios than during 
the requested ITP scenarios (p < 0.0001). 

As mentioned previously, execution errors occurred during 29 of the requested ITP scenarios. However, 
since it was possible for a subject pilot to make multiple errors during a single scenario, a total execution error 
count greater than 29 occurred. Specifically, 33 execution errors occurred during the requested ITP scenarios, 
and, as stated above, nine of the ITP execution errors were identified as being safety-related. Two of the safety- 
related ITP execution errors involved instances in which subject pilots identified a leading aircraft as a following 
aircraft when communicating their desire to perform an ITP flight level change to ATC. Although it was also 
possible for a subject pilot to incorrectly identify a following aircraft as a leading aircraft during a 
communication with ATC, this type of error did not occur. Two safety-related ITP execution errors occurred 
when subject pilots failed to include the third number of a reference aircraft’s call sign in their requests to make 
an ITP flight level change, and three safety-related ITP execution errors occurred when subject pilots referenced 
an incorrect aircraft. When referencing an incorrect aircraft, the subject pilots transposed or incorrectly entered 
one or two of the letters in the three-letter airline code. For example, one subject pilot entered “DHL615” rather 
than “DLH615.” In another case a subject pilot entered “DAL615” instead of “DLH615.” The remaining two 
safety-related ITP execution errors occurred when one subject pilot failed to adhere to the aircraft performance 
criteria required during two different requested ITP scenarios. In one instance, the subject pilot failed to 
maintain his required Mach number. The subject made a speed change of 0.01 Mach that allowed a maneuver 
that otherwise would not have been possible. In the other instance, the same subject pilot failed to maintain 
required vertical speed. The subject pilot entered a 1,000 foot altitude change via the desktop simulator’s MCP 
resulting in a positive vertical speed over a span of 10 seconds during which the aircraft climbed approximately 
20 ft while performing a descent maneuver. While a 20 ft deviation in the opposite direction is not operationally 
significant from a vertical separation point of view, it would result in extending the overall time required to 
complete the flight level change and could thus potentially result in a loss of longitudinal separation. 

Twenty-four non-safety-related execution errors occurred during the scenarios involving ITP flight level 
changes. Sixteen of these ITP execution errors involved instances in which subject pilots referenced an aircraft 
at the desired flight level during a flight level change request; three errors involved subject pilots requesting a 
climb or descent to an incorrect flight level; three errors were related to syntax errors in the ITP phraseology; 
one error occurred when a subject pilot failed to comply with the ATC instruction to “report reaching” when 
level at the completion of the approved maneuver (per ICAO procedures for oceanic/remote flight operations); 
and one ITP execution error involved a subject pilot failing to acknowledge ATC’s denial of an ITP flight level 
change request (per the briefed standard procedures for data-linked ATC communications) before making a 
second ITP flight level change request. 

In addition to examining the percentages of requested standard and ITP scenarios involving the occurrence of 
at least one execution error or safety-related execution error, the distribution of execution errors (both safety- 
related and non-safety-related) was examined according to subject pilot, type of flight level change maneuver, 
ITP geometry, direction of flight level change maneuver (i.e., climb versus descent), achievability of flight level 
change maneuver, and direction of flight (i.e., Eastbound versus Westbound). Since execution errors did not 
occur during the requested standard scenarios, requested standard climb and descent maneuvers are discussed 
only in conjunction with the distribution of execution errors across the different types of flight level change 
maneuvers. Specific types of flight level change maneuvers are not discussed in conjunction with the 
distribution of execution errors across individual subject pilots. Requested ITP flight level change maneuvers 
are discussed in conjunction with the distribution of execution errors across ITP geometries, direction of flight 
level change maneuvers, achievability of flight level change maneuvers, and directions of flight. 
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ITP Execution Errors by Subject Pilot 

The distribution of ITP execution errors across individual subject pilots is presented in Figure 17. In this 
figure, groups of subject pilots who participated in the experiment during the same two day period are indicated 
by the vertical lines within the area delineated by the x and y axes. For example, Subject Pilots #1 and #2 
participated in the experiment together, and Subject Pilots #3, #4, and #5 participated in the experiment together. 
Note that although at least three subject pilots always participated in the experiment together, data from some 
subject pilots were not usable due to simulation glitches and/or data recording errors, and these subject pilots are 
not shown in the figure (nor were their data analyzed or reported in this paper). 



Figure 17. Total number of execution errors associated with individual subject pilots 


As shown in Figure 17, two subject pilots (i.e., #6 and #10) did not commit any execution errors; and of the 
10 subject pilots that did commit errors, nine committed five or less. Six subject pilots committed safety-related 
execution errors, but four of these subject pilots committed only a single safety-related error. One subject pilot 
committed two safety-related execution errors, and one subject pilot committed three safety-related execution 
errors. One subject pilot committed nine execution errors, but none of his errors were safety-related. Since 
individual differences are expected to occur among research participants, no statistical analysis was performed 
in conjunction with the execution error rates associated with individual subject pilots. 

Cochran’s Q Tests and McNemar Tests (nonparametric within- subject tests appropriate for analyzing related 
samples of nominal data) [12] [13] were performed in conjunction with the total number of execution errors 
associated with the different types of flight level change maneuvers, ITP geometries, directions of flight level 
change maneuver, achievability of flight level change maneuvers, and directions of flight. Graphical 
presentations of the percentages of requested scenarios involving safety-related execution errors versus non- 
safety-related execution errors are provided, and total numbers of safety-related execution errors versus non- 
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safety-related execution errors and percentages of requested scenarios involving safety-related execution errors 
versus non-safety-related execution errors are presented in tabular form as well. 


Execution Errors by Type of Flight Level Change Maneuver 

The distribution of execution errors across the different types of flight level change maneuvers is presented 
in Figure 18 (means associated with different capital letters are significantly different in McNemar Tests at p < 
0.05). Within this graph, the numbers in parentheses shown with the x-axis’ labels of flight level change 
maneuver type correspond to the number of requested scenarios calculated using the data presented in Table 7. 
Within Table 7, The “Total Number of Scenarios” corresponds to the “Total Number of Test Conditions 
involving a Particular Type of Flight Level Change Maneuver.” 
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Figure 18. Percentages of requested types of flight level change maneuvers that had at least one execution error 
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Table 7. Expected versus Requested Type of Flight Level Change Maneuver 



Note: “SC” = Standard Climb; “SD” = Standard Descent; “LC” = ITP Leading Climb; “LD” = ITP Leading Descent; “FC” 
= ITP Following Climb; “FD” = ITP Following Descent; “CC” = ITP Combined Climb; and “CD” = ITP Combined 
Descent. 


Total numbers of safety-related execution errors and non-safety-related execution errors are presented in 
Table 8. 

Table 8. Distribution of Execution Errors across Different Types of Requested Flight Level Change Maneuvers 


Requested 

Flight Level Change 

Maneuver 

Number of Non-Safety- 
Related Execution 
Errors 

Number of Safety- 
Related Execution 
Errors 

Total Number of 
Execution Errors 

Leading Climb 

6 

2 

8 

Leading Descent 

8 

1 

9 

Following Climb 

1 

0 

1 

Following Descent 

0 

2 

2 

Combined Climb 

7 

2 

9 

Combined Descent 

2 

2 

4 


A Cochran’s Q Test was performed to determine if a significant difference existed among the total number of 
execution errors that occurred during the requested types of flight level change maneuvers. Since this test 
revealed a significant difference between the number of execution errors that occurred during at least two of the 
requested types of flight level change maneuvers (Q[7] = 27.3974; p = 0.0003), McNemar Tests were performed 
to determine which of the requested types of flight level change maneuvers were associated with significantly 
different numbers of execution errors. With respect to statistically significant differences revealed by the 
McNemar Tests (p < 0.05), it was found that: 
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Significantly fewer execution errors occurred during the requested standard climb maneuvers and the 
requested standard descent maneuvers than during the requested leading climb maneuvers, the requested 
leading descent maneuvers, the requested combined climb maneuvers, and the requested combined 
descent maneuvers; and 

Significantly fewer execution errors occurred during the requested following climb maneuvers than 
during the requested leading climb maneuvers, the requested leading descent maneuvers, and the 
requested combined climb maneuvers. 

With respect to non-significant differences revealed by the McNemar Tests (p > 0.05), it was found that, 
statistically speaking: 

The same number of execution errors occurred during the requested standard climb maneuvers, the 
requested standard descent maneuvers, the requested following climb maneuvers, and the requested 
following descent maneuvers; 

The same number of execution errors occurred during the requested following climb maneuvers, the 
requested following descent maneuvers, and the requested combined descent maneuvers; and 

The same number of execution errors occurred during the requested following descent maneuvers, the 
requested leading climb maneuvers, the requested leading descent maneuvers, the requested combined 
climb maneuvers, and the requested combined descent maneuvers. 

Execution Errors by ITP Geometry 

As described in the “In-Trail Procedure” section of this document, the ITP geometries included leading 
maneuvers, following maneuvers, and combined maneuvers. The reference aircraft is located behind the ITP 
Aircraft during a leading maneuver, and the reference aircraft is located ahead of the ITP Aircraft during a 
following maneuver. During a combined maneuver, which involves both a leading and a following reference 
aircraft, there is a reference aircraft located ahead of and behind the ITP Aircraft. Due to subject pilot selection 
errors, there were also two requested ITP scenarios that involved a combined maneuver with two leading 
reference aircraft instead of one leading and one following reference aircraft. During these two requested ITP 
scenarios, both reference aircraft were located behind the ITP Aircraft. 

The distribution of execution errors across the different ITP geometries is presented in Figure 19 (means 
associated with different capital letters are significantly different in McNemar Tests at p < 0.05). Within this 
graph, the numbers in parentheses shown with the x-axis’ labels of ITP geometries correspond to the number of 
requested scenarios that occurred during the experiment. Table 9 presents the data used to calculate the number 
of requested leading, following, and combined flight level change maneuvers. Within Table 9, the “Total 
Number of Scenarios” corresponds to the “Total Number of Test Conditions involving a Particular ITP 
Geometry.” Total numbers of safety-related execution errors and non-safety-related execution errors are 
presented in Table 10. 
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Figure 19. Percentages of requested ITP geometries that had at least one execution error 


Table 9. Expected versus Requested ITP Geometries 



ITP Geometries 

Subject Pilot 

Expected 

Combined 

Requested 

Combined 

Expected 

Following 

Requested 

Following 

Expected 

Leading 

Requested 

Leading 

1 

4 

6 

4 

3 

4 

5 

2 

4 

5 

4 

4 

4 

3 

3 

4 

5 

4 

4 

4 

4 

4 

4 

5 

4 

3 

4 

5 

5 

4 

4 

4 

4 

4 

4 

6 

4 

4 

4 

4 

4 

4 

7 

4 

4 

4 

4 

4 

4 

8 

4 

4 

4 

4 

4 

4 

9 

4 

4 

4 

4 

4 

4 

10 

4 

4 

4 

4 

4 

4 

11 

4 

4 

4 

4 

4 

4 

12 

4 

4 

4 

4 

4 

5 

Total Number 
of Scenarios 

48 

53 

48 

46 

48 

50 


Table 10. Distribution of Execution Errors across Different Types of Requested ITP Geometries 


Requested 
ITP Geometry 

Number of Non-Safety- 
Related Execution 
Errors 

Number of Safety- 
Related Execution 
Errors 

Total Number of 
Execution Errors 

Combined 

9 

4 

13 

Following 

1 

2 

3 

Leading 

14 

3 

17 
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A Cochran’s Q Test was performed to determine if a significant difference existed among the total number of 
execution errors that occurred during the requested ITP geometries. Since this test revealed that a significant 
difference existed between the number of execution errors that occurred during at least two of the requested ITP 
geometries (Q [2] = 19.5000; p = 0.0001), McNemar Tests were performed to determine which of the requested 
ITP geometries were associated with significantly different numbers of execution errors. The McNemar Tests 
revealed that significantly fewer execution errors occurred during the requested following maneuvers than 
during the requested leading maneuvers (p = 0.0005) and that significantly fewer execution errors occurred 
during the requested following maneuvers than during the requested combined maneuvers (p = 0.0039). 
Statistically speaking, an equivalent number of execution errors occurred during the requested leading 
maneuvers and the requested combined maneuvers (p = 0.2500). At this point, it is not clear why the error rates 
are lower for following maneuvers than for leading maneuvers. A possible explanation for the higher error rates 
with combined maneuvers would be the additional complexity of including a second reference aircraft in an ITP 
request, but at this point, without further data, such an explanation is little more than conjecture. 

Execution Errors by Direction of Flight Level Change Maneuver 

The distribution of execution errors associated with the different directions of the flight level change 
maneuvers (i.e., climbs versus descents) is presented in Figure 20. Within this graph, the numbers in 
parentheses shown with the x-axis’ labels of the flight level change maneuver direction types correspond to the 
number of requested scenarios calculated using the data presented in Table 11. Within Table 11, the “Total 
Number of Scenarios” corresponds to the “Total Number of Test Conditions involving a Particular Flight Level 
Change Maneuver Direction.” Total numbers of safety-related execution errors and non-safety-related 
execution errors are presented in Table 12. 
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Figure 20. Percentages of requested flight level change maneuvers involving climbs versus descents that had at least one 
execution error 
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Table 11. Expected versus Requested Flight Level Change Maneuvers Involving Climbs versus Descents 



Climb Scenarios 

Descend Scenarios 

Subject Pilot 

Expected 

Requested 

Expected 

Requested 

1 

6 

7 

6 

7 

2 

6 

6 

6 

6 

3 

6 

7 

6 

6 

4 

6 

7 

6 

6 

5 

6 

7 

6 

5 

6 

6 

6 

6 

6 

7 

6 

6 

6 

6 

8 

6 

6 

6 

6 

9 

6 

6 

6 

6 

10 

6 

6 

6 

6 

11 

6 

6 

6 

6 

12 

6 

7 

6 

6 

Total Number of 
Scenarios 

72 

77 

73 

72 


Table 12. Distribution of Execution Errors associated with Different Requested Flight Level Change Maneuver Directions 
(i.e., Climbs versus Descents) 


Requested 
Flight Level Change 
Maneuver Direction 

Number of Non-Safety- 

Number of Safety-Related 

Total Number of 

Related Execution Errors 

Execution Errors 

Execution Errors 

Climb 

14 

4 

18 

Descent 

10 

5 

15 


A McNemar Test was performed to determine if a significant difference existed between the total number of 
execution errors that occurred during requested flight level change maneuvers involving climbs and those 
involving descents. This test revealed that, statistically speaking, an equivalent number of execution errors 
occurred during the requested flight level change maneuvers involving climbs and the requested flight level 
change maneuvers involving descents (p = 0.0625). 

Execution Errors by Achievability of Flight Level Change 

The distribution of execution errors associated with the achievability of flight level changes (i.e., achievable 
versus unachievable) is presented in Figure 21. Within this graph, the numbers in parentheses shown with the x- 
axis’ labels correspond to the number of requested scenarios calculated using the data presented in Table 13. 
Within Table 13, the “Total Number of Scenarios” corresponds to the “Total Number of Test Conditions 
involving the Achievability of Flight Level Change.” Total numbers of safety-related execution errors and non- 
safety-related execution errors are presented in Table 14. 
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Figure 21. Percentages of requested flight level change maneuvers that were achievable or unachievable and had at least 
one execution error 


Table 13. Expected versus Requested Types of Flight Level Change Maneuver Achievabilty 



Achievable Flight Level 
Change Maneuver 

Unachievable Flight Level 
Change Maneuver 

Subject Pilot 

Expected 

Requested 

Expected 

Requested 

1 

6 

8 

6 

6 

2 

6 

6 

6 

6 

3 

6 

7 

6 

6 

4 

6 

7 

6 

6 

5 

6 

7 

6 

5 

6 

6 

6 

6 

6 

7 

6 

6 

6 

6 

8 

6 

6 

6 

6 

9 

6 

6 

6 

6 

10 

6 

6 

6 

6 

11 

6 

6 

6 

6 

12 

6 

7 

6 

6 

Total Number 

72 

78 

72 

71 

of Scenarios 
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Table 14. Distribution of Execution Errors across Different Requested Flight Level Change Maneuver Achievabilities 


Requested Type of 
Flight Level Change 
Maneuver 
Achievability 

Number of Non-Safety- 
Related Execution Errors 

Number of Safety-Related 
Execution Errors 

Total Number of 
Execution Errors 

Achievable 

15 

6 

21 

Unchievable 

9 

3 

12 


A McNemar Test was performed to determine if a significant difference existed between the total number of 
execution errors that occurred during the requested flight level change maneuvers that were achievable as 
opposed to unachievable. This test revealed that significantly fewer execution errors occurred during the 
requested flight level change maneuvers that were unachievable than during the requested flight level change 
maneuvers that were achievable (p = 0.0020). 

A partial explanation for the higher error rates observed during the achievable flight level change maneuvers 
is that the unachievable maneuvers left pilots with no opportunity to commit errors in the actual performance of 
the flight level change, e.g., failure to maintain cruise Mach or minimum vertical speed. This explanation 
accounts for two of the additional three safety-related execution errors incurred during the achievable maneuver 
scenarios, but cannot account for any of the additional non-safety related errors. The reasons for these 
differences are currently unknown. 

Execution Errors by Direction of Flight 

The distribution of execution errors associated with the two directions of flight (i.e., Eastbound versus 
Westbound) is presented in Figure 22. Within this graph, the numbers in parentheses shown with the x-axis’ 
labels of direction of flight correspond to the number of requested scenarios calculated using the data presented 
in Table 15. Within Table 15, the “Total Number of Scenarios” corresponds to the “Total Number of Test 
Conditions involving a Particular Direction of Flight.” Total numbers of safety-related execution errors and 
non- safety-related execution errors are presented in Table 16. 
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Figure 22. Percentages of requested scenarios involving Eastbound versus Westbound flights that had at least one execution 

error 


Table 15. Expected versus Requested Direction of Flight 



Eastbound 


Westbound 


Subject Pilot 

Expected 

Requested 

Expected 

Requested 

1 

6 

6 

6 

8 

2 

6 

6 

6 

6 

3 

6 

7 

6 

6 

4 

6 

7 

6 

6 

5 

6 

6 

6 

6 

6 

6 

5 

6 

7 

7 

6 

5 

6 

7 

8 

6 

5 

6 

7 

9 

6 

6 

6 

6 

10 

6 

8 

6 

4 

11 

6 

6 

6 

6 

12 

6 

7 

6 

6 

Total Number 

72 

74 

72 

75 

of Scenarios 


Table 16. Distribution of Execution Errors across Requested Directions of Flight 


Requested 
Direction of Flight 

Number of Non-Safety- 
Related Execution Errors 

Number of Safety-Related 
Execution Errors 

Total Number of 
Execution Errors 

Eastbound 

12 

3 

15 

Westbound 

12 

6 

18 
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A McNemar Test was performed to determine if a significant difference existed between the total number of 
execution errors that occurred during the requested Eastbound versus Westbound flights. This test revealed that, 
statistically speaking, an equivalent number of execution errors occurred during the flights traveling Eastbound 
and the flights traveling Westbound (p = 0.2500). 

Discussion of Execution Errors 

The fact that no execution errors occurred during standard flight level change scenarios, while errors 
occurred in over 18% of ITP scenarios, can be neither ignored nor minimized. Of at least equal significance is 
the fact that no safety-related execution errors occurred during standard flight level change scenarios but 
occurred in over 6% of ITP flight level change scenarios. The specific safety-related execution errors were 
described previously, but collectively they represent errors that potentially cannot be detected by ATC or by 
other means and that could result in a loss of separation between aircraft. Each of the execution error types 
committed, beginning with the safety-related execution errors, will be discussed in this subsection along with 
possible procedure or display design changes that might reduce or eliminate the errors. 

Seven of the nine safety-related execution errors involved subject pilots sending incorrect information in 
their ITP request that could not necessarily be detected by the ground controller. These included two errors 
where subject pilots identified a leading aircraft as a following aircraft, three errors where the subject pilots 
entered an incorrect call sign for a reference aircraft, and two errors where subject pilots left off the third 
number of a reference aircraft’s call sign (in effect, another form of incorrect call sign). These errors might be 
mitigated through a change in the Oceanic Operations application display design and/or the ATC data-link 
interface. For example, if the display and data-link interface were interfaced in such a way that the pilot only 
needs to select the desired reference aircraft (e.g., by “clicking” on the displayed image of the reference aircraft) 
rather than having to compose an ITP text message (e.g., “ITP L/UAL123/25”), then most of these operator 
entry errors might be eliminated. The disadvantage of such a system is that it would require expensive changes 
to certified aircraft systems such as the data-link communications interface. A less comprehensive but also less 
expensive alternative might be to redesign the current display so that the data tag includes the appropriate “L /” 
or “F /” prefix in front of the call sign, which might reduce or eliminate the leading/following errors. Lastly, the 
development of crew crosscheck procedures, where one crewmember checks the other’s data entry before the 
ITP request is sent, would likely reduce or eliminate these safety-related errors. 

The remaining two safety-related execution errors involved failure to maintain cruise Mach, or appropriate 
vertical speed, during the ITP flight level change. The latter case apparently involved either pilot error or 
“tinkering” while operating the autoflight system (it was unclear which was the case), and in another 
circumstance could just as likely have occurred during a standard flight level change. The design implication, 
though, is that the ITP safety analyses must assume that such errors or events will occur from time to time, and 
assure that design vertical speeds, distances, ground speed differentials, etc. for ITP are sufficient to assure the 
necessary levels of safety in the presence of such pilot errors. The former case (failure to maintain cruise Mach) 
is interesting in that the error was intentional; that is, the subject pilot “gamed” the scenario by intentionally 
changing ownship speed by 0.01 Mach to allow an ITP that would not otherwise have been possible (at the 
original Mach, the reference aircraft would have been too close before ATC could have responded to the subject 
pilot’s ITP request, so the reassessment would have failed and the ITP clearance would have had to be refused). 
The possibility of pilots “gaming” such situations was widely discussed both prior to the experiment by the 
researchers, and during the debrief sessions with the subject pilots, and the consensus was that such behavior 
would inevitably occur occasionally, at some unknown frequency. When subject pilots were asked if they 
thought such speed changes would be hazardous to safety, they generally responded in the negative. However, 
when it was noted that their aircraft might already be a reference aircraft for another in-progress ITP when they 
chose to alter their speed, some changed their minds on the safety implications. To summarize, without further 
study, the safety effects of such intentional “gaming” by pilots is unknown. 
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Sixteen of the 24 non-safety-related execution errors (i.e., two-thirds of these errors) involved subject pilots 
incorrectly referencing aircraft at the desired, rather than the intervening, flight level. This type of error was 
committed by slightly more than half of all the subject pilots (seven of 12). The error occurred in all six of the 
expected standard scenarios where subject pilots inappropriately requested an ITP (i.e., committed selection 
error), as there was no observable aircraft available for them to reference on the intervening flight level. It also 
occurred in seven of the eight scenarios involving selection errors where subject pilots requested an 
inappropriate ITP geometry; that is, in these cases, rather than requesting either a leading or following ITP as 
appropriate for a particular scenario, subject pilots made a combined leading-following ITP request by 
incorrectly referencing an “extra” aircraft at the desired flight level in addition to the one appropriately 
referenced at the intervening flight level. The remaining three occurrences of this error were made separately by 
three different subject pilots but all in the same scenario - an ITP leading descent scenario with observable 
blocking traffic at the desired flight level, so the subject pilots were expected to make no request. Not only did 
these subject pilots request an ITP descent referencing an aircraft at the desired flight level, but they requested 
an ITP descent of only 1,000 ft when, by design, ITP flight level changes are always at least 2,000 ft. (i.e., they 
committed a second execution error). 

Why the above-described errors occurred (i.e., referencing aircraft at the desired flight level), and especially 
why they occurred in such large numbers, is a source of significant puzzlement and speculation among the 
authors. All of the subject pilots had received prior exposure to the ITP concepts via the survey they had 
completed, and during the experiment training they received explicit and repeated instruction that only aircraft 
on intervening flight level(s) could be referenced in an ITP request. In fact, when it became apparent halfway 
through the experiment sessions that this error was occurring in large numbers, the researchers broke with the 
usual practice of keeping training identical across all experiment sessions and emphasized this concept even 
more strongly with the second half of the subject pilots. The results were unchanged, however, as this error 
continued to occur in significant numbers. During the debrief sessions with this second half of the subject 
pilots, the fact that these errors were repeatedly occurring among multiple subject pilots was brought up, and the 
subject pilots were asked for possible explanations. The answers varied but basically came down to three 
responses: either the subject pilots didn’t realize they had made the error and couldn’t say why they had done so; 
did it by mistake and recognized afterward that they had done so in error; or intentionally did it to let ATC know 
that, for example, they “knew about that aircraft.” 

This last response has an analogy in current-day operations with direct voice communication in visual flight 
conditions, when ATC will call out the position of a nearby aircraft and a pilot will answer that he or she “has 
them on TCAS” (as opposed to the proper responses of either “traffic in sight” or “negative contact”). The fact 
that the crew sees the called traffic on their TCAS display has no operational significance to the controller - that 
is, the controller cannot delegate separation by instructing the pilot to maintain visual separation with traffic 
“seen” only on TCAS - but pilots continue to respond in this way to let ATC know that they “know about that 
aircraft” (and therefore imply that they may soon acquire it visually). It is possible that this same mentality is 
causing some subject pilots to incorrectly reference aircraft at the desired flight level in an ITP request. 

Changes to the design of the Oceanic Operations application’s traffic display might mitigate the error of 
pilots referencing aircraft at the desired flight level. For example, the display could be designed so as to remove 
call signs from the data tags of aircraft at other than intervening flight levels, preventing pilots from mistakenly 
referencing them in an ITP request. Alternatively, errors of these types might be minimized by adding decision 
support to the Oceanic Operations application, so that “allowable” reference aircraft are highlighted or otherwise 
made apparent to pilots, and other aircraft are de-emphasized for use as reference aircraft. Similarly, the ITP 
could be changed procedurally such that ATC is instructed to always deny any ITP requests with 
inappropriately-referenced aircraft, and if pilots were informed of this in their ITP training they might check 
more rigorously for this error prior to sending an ITP request. Lastly, the error could be treated by ATC as 
“harmless” superfluous information and ignored when evaluating the received ITP request. 
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On a broader note, the ATC knowledge and experience level of the pilots participating in the experiment and 
in the airline pilot population at large must inevitably be considered in light of the non-safety-related execution 
errors discussed in the previous paragraphs. While airline pilots are highly trained and experienced in their 
respective flight crew tasks, and while some have doubtless gained some knowledge of air traffic control 
procedures at “the other end of the microphone” through their flight experience or independent study, the fact 
remains that in general they receive little or no formal training in ATM and separation concepts, standards, and 
techniques as part of their airline pilot training. The subject pilots in the experiment were generally learning 
about ground-side ATM and separation concepts and techniques as well as active airborne involvement in new 
ATM procedures (i.e., the ITP), for essentially the first time, before being asked to appropriately select and 
perform the procedures. While the ITP is relatively simple in concept, in its current form it can be applied in a 
diverse set of traffic conditions and geometries, and does require pilots to exercise some air traffic separation 
judgment and proper application of the ITP concepts in each of these diverse sets of conditions and geometries. 
In general, it may be that some level of conceptual training on ATM separation standards is required to 
minimize errors of the type discussed in the previous paragraphs, or that 1) the applicability of the ITP is limited 
to well-defined simple geometries; 2) specific rules-of-thumb (versus the current judgment-based guidance) as 
to when pilots should request an ITP are defined; and/or 3) the level of automation/decision support provided by 
the Oceanic Operations application is increased. As an example, the Oceanic Operations application could be 
enhanced to construct the appropriate ITP request string, if any, based on built-in rules-of-thumb and the pilots’ 
selection of the desired flight level. 

Subjective Assessments of the ITP’s Validity 

With respect to the ITP’s validity, subject pilots used the post-scenario questionnaire, the post-experiment 
questionnaire, and the post-experiment group debrief sessions to describe their impressions of the correctness, 
completeness, appropriate specification, and logical sequencing of the procedural steps that they were instructed 
to use while executing ITP maneuvers. Subject pilots described their ITP performance concerns through the use 
of the post-experiment questionnaire and during the post-experiment group debrief sessions. These subjective 
impressions and concerns are provided in the following paragraphs, as are additional subjective impressions and 
concerns regarding acceptability and workload in subsequent sub-sections. In most cases these subject pilots’ 
assessments are presented without editorial comment from the authors, except where necessary for clarification, 
in keeping with the concept that much of the value of these subjective assessments lies in the unedited 
impressions of current-day subject matter experts (i.e., oceanic line pilots) after initial exposure to ITP concepts 
and technologies in their present form. 

Through their post-scenario questionnaire responses, subject pilots indicated in 92.7% of their responses that 
the procedural steps used to execute ITP maneuvers were correct, complete, and appropriately specified and 
indicated in 94.3% of their responses that the procedural steps occurred in a logical sequence. No particular 
type of ITP maneuver was characterized as being associated with procedural steps that were found to be 
problematic. Although it was hypothesized that subject pilots would not find any missing, incomplete, or 
extraneous procedural steps associated with the ITP, this was not the case. With respect to procedural steps 
found to be “missing” from the ITP Checklist, one subject pilot stated that the checklist should include an 
additional “Verify Message” step to be completed by the Pilot Flying (PF). This additional step should come 
after the “Prepare” and before the “Send” step is completed. It was suggested that this checklist modification 
would help pilots “avoid syntax errors” and provide an additional “independent check.” This same subject pilot 
also recommended that it might be more appropriate to execute the checklist’s “Reassess ITP Criteria” step 
using three independent steps, as is currently done with CPDLC clearances. These steps would occur as 
follows: 1) PNF pulls up message and prints it; 2) PNF silently reads message and reassesses situation; and 3) 
PF silently reads message and reassesses situation. Another subject pilot suggested including an additional 
procedural step (not specified) that would facilitate subsequent requests for alternatives to improve performance 
options when an initial “optimum/desired/recommended” request is denied by ATC. This same subject pilot 
indicated that due to the fact that the procedure includes a checklist, it should outline a means to cancel a request 
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if ITP criteria are not met in the time between requesting and receiving an ATC clearance. The pilot also 
indicated that the ITP Checklist should include an “Arm” or “Report Reaching Assigned Altitude” step. 

Using the post-experiment questionnaire, the subject pilots commented on their use of the procedural steps 
outlined in the ITP Checklist and described any performance concerns that they had with the ITP. According to 
these responses, two of the subject pilots used the ITP Checklist during every ITP maneuver while, on average, 
the other 10 subject pilots reported using the checklist during 37% (SD = 17.4%) of the ITP maneuvers. In 
general, the subject pilots that used the checklist intermittently indicated that they did not need to refer to the 
checklist during every ITP maneuver since the procedures were “intuitively obvious.” When asked if they 
reassessed the ITP criteria every time they received an ITP clearance, as required by the checklist, eight of the 
subject pilots stated that they always performed the required reassessment while four of the subject pilots stated 
that they did not adhere to this procedural step. Two subject pilots indicated that they only reassessed the ITP 
criteria when distance or speed requirements were “nearing the limits.” With respect to identifying potential 
performance related ITP issues, six of the subject pilots reported that they had ITP performance concerns. For 
example, a question was posed regarding the possibility of the ITP Aircraft encountering wake turbulence 
during an ITP following maneuver and the possibility of the reference aircraft encountering wake turbulence 
during an ITP leading maneuver. Also, the impact of non-participating aircraft (reference or non ADS-B 
aircraft) on the success or failure of the ITP was identified as being a “significant factor.” Lastly, it was 
mentioned that an emphasis will need to be placed on constant Mach operations since current day operations in 
regions other than NATOTS, heavily rely on the FMC’s “econ” setting. This FMC setting adjusts the Mach to 
the most efficient fuel or time operation and as a result is not associated with constant Mach. 

Subject pilots were also encouraged to comment on their perceptions of the ITP’s validity during the post- 
experiment group debrief sessions. With respect to the correctness, completeness, and appropriate specification 
of the procedural steps used to execute ITP maneuvers, one item of interest was identified by the subject pilots 
as missing. Specifically, it was noted that the need to consult winds information was not addressed within the 
ITP. However, the ITP was designed such that pilots do not need to consult winds information before making 
an ITP request. When commenting on their use of the ITP Checklist, several subject pilots stated that, while the 
checklist should be made available to flight crews, it will probably only be used during the procedure’s initial 
implementation; subject pilots expressed the opinion that flight crews are not likely to refer to the checklist once 
they are familiar with the ITP, given that the procedure is relatively “simple.” When asked to describe their ITP 
performance concerns, the subject pilots mentioned two issues during the post-experiment group debrief 
sessions. First, the potential consequences associated with failures of aircraft to comply with ITP performance 
requirements were identified as a cause for concern. Second, the existence of wake vortices during the 
execution of an ITP maneuver was discussed as a potential concern; however, it was acknowledged that the 
impact of wake vortices must also be considered during current day operations. 

Subjective Assessments of the ITP’s Acceptability 

Subject pilots used the post-scenario questionnaire and the post-experiment group debrief sessions to 
comment on the ITP’s acceptability and to describe alternate ways in which they might have performed a given 
flight level change maneuver. Subject pilots used the post-experiment questionnaire and the post-experiment 
group debrief sessions to comment on issues associated with the general acceptability of the ITP and to describe 
the ways in which the ITP could be potentially beneficial to flight crews, passengers, and airline companies. 
The workload level associated with the performance of the ITP is directly related to the procedure’s 
acceptability; however, the subject pilots’ subjective assessments of workload are discussed in a subsequent 
section. 

When asked via the post-scenario questionnaire to document their suggestions for performing a given flight 
level change differently, two subject pilots reiterated the opinion of another subject pilot (mentioned previously 
in conjunction with “Subjective Assessments of the ITP’s Validity”) who stated that he would like for a 
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procedural step to be added to the ITP’s Checklist to facilitate subsequent flight level change requests when an 
initial request is denied by ATC. These additional two subject pilots indicated that the ITP’s acceptability 
would increase if pilots were afforded the opportunity to communicate alternate altitude requests to ATC in 
hopes of securing approval for a flight level change should their initial requests be denied. One subject pilot 
provided this comment in response to a standard climb scenario involving a request that was rejected by ATC, 
and the other subject pilot expressed this opinion in terms of a more general, overarching preference. Another 
subject pilot indicated that the ITP would be more acceptable if its safety distance and speed tolerances were 
reduced (however, these criteria are directly tied to safety analyses used to guide the design of the ITP [2]). 
While completing post-scenario questionnaires associated with ITP maneuvers that could not be completed due 
to a failure to maintain ITP speed requirements (e.g., “Unable due to Ground Speed” scenarios), two subject 
pilots reported that, given the chance during actual flight operations, they would alter the speed of their aircraft 
to maintain ITP spacing long enough to complete a flight level change maneuver. These subject pilots indicated 
that the acceptability of the ITP would be increased if the requirement for constant Mach was eliminated. 

When asked via the post-experiment questionnaire if, given their operational experience, they thought of the 
ITP as being an acceptable procedure, all 12 subject pilots answered “yes.” But, two subject pilots mentioned 
concerns regarding the procedure’s reliance on an “honor system” in which the distances among aircraft must be 
accurately reported to ATC. The ITP free text phraseology used to communicate with ATC during the 
experiment was characterized by all 12 subject pilots as being acceptable in that it was “easy to understand and 
use.” With respect to their impressions regarding the level of safety associated with performing the ITP as 
compared with current day procedures, the subject pilots’ responses were somewhat mixed. One subject pilot 
characterized the ITP as being “less safe than” current day procedures but stated that the ITP is still 
“procedurally sound”; seven subject pilots characterized the ITP as being “equally as safe as” current day 
procedures; and four subject pilots characterized the ITP as being “safer than” current day procedures. The 
subject pilots that characterized the ITP as being “safer than” current day procedures stated that the ITP display: 
is similar to onboard weather radar in than the operator gets to see more of the current operating environment; 
provides “eyes” in IMC and enhanced SA; provides more SA than the TCAS does; and reduces 
misinterpretations of clearances. The enhanced SA was considered to be a result of the display implementation 
and not the procedure. 

Since it was hypothesized that subject pilots would find the ITP to be beneficial in situations where climbs or 
descents would not otherwise be possible, the post-experiment questionnaire was used to elicit subject pilots’ 
comments regarding potential benefits that the ITP might have for flight crews, passengers, and airline 
companies. For flight crews, the subject pilots identified potential benefits as being: improved traffic SA; more 
efficient aircraft operation (resulting from the ability to manage climbs at efficient times); enhanced job 
satisfaction associated with the opportunity to achieve optimum performance; and improvements in safety due to 
the avoidance of weather and turbulence. For passengers, the subject pilots identified potential benefits as 
being: smoother rides due to the flight crew’s ability to vacate turbulent altitudes; improved ticket prices if 
airline companies experience savings (see below) that are passed on to the consumer; and increased instances of 
on-time arrivals since delays due to oceanic congestions may be avoided, more fuel may be available to hold at a 
destination thereby improving passengers’ chances of making connecting flights, and having more fuel at the 
destination lowers the likelihood of required diversions. For airline companies, the subject pilots identified 
potential benefits as being: fuel savings; safety and passenger comfort; and improved scheduling due to route 
planning. 

The post-experiment questionnaire also presented subject pilots with a “what if’ type question whose 
purpose was to collect their thoughts regarding off-nominal conditions, even though off-nominal conditions 
were outside the scope of the experiment. Specifically, subject pilots were asked the following question: “If you 
encountered off-nominal conditions while performing an ITP flight level change (e.g., you observe the reference 
aircraft deviating), how easy would it be for you to revert back to the use of standard regional contingency 
procedures?” Four subject pilots indicated that it would be “very easy” to revert to standard regional 
contingency procedures if an off-nominal situation was encountered during the performance of an ITP 
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maneuver; five subject pilots indicated that it would be “easy” to revert back to standard regional contingency 
procedures; two subject pilots were undecided as to how easily they could revert back to standard regional 
contingency procedures; and one subject pilot indicated that it would be “somewhat difficult” to revert back to 
standard regional contingency procedures; this last subject pilot stated that “[t]here needs to be a defined 
procedure” in place for flight crews to use when off-nominal situations are encountered. (It should be noted that 
during their training, subject pilots were told that in actual implementation that regional contingency procedures 
would be followed for off-nominal situations; but, for the experiment, they could notify an experimenter if an 
off-nominal situation was encountered during the experiment.) 

During the post-experiment group debrief sessions, subject pilots were asked to comment on the general 
acceptability of the ITP and to describe any ways in which they might have preferred to perform flight level 
change maneuvers. Three alternate, or more acceptable, methods for performing flight level change maneuvers 
were discussed. First, it was suggested that a definitive reason be provided to flight crews via the ITP display 
whenever a flight level change request is denied as this would enhance crews’ SA. Second, it was suggested 
that ATC informs flight crews or issues “auto-clearances” as soon as the traffic aircraft associated with previous 
denials were no longer “blocking” a requested flight level change. Third, it was suggested that speed change 
requests be allowed and that ATC be notified of such requests whenever the 0.01 Mach buffer range was 
exceeded. 

Subject pilots’ comments regarding the acceptability of the ITP centered around four primary topics during 
the post-experiment group debrief sessions: overall acceptability, safety, phraseology, and ATC’s role in 
separation assurance. In support of the procedure’s overall acceptability, several subject pilots said that they 
would like to see the ITP implemented for use during actual oceanic flight operations and were interested in 
knowing how quickly this might happen. Subject pilots’ made comments such as “[i]t’s time to shrink the 
airspace between aircraft” and that “too much separation [between aircraft is] required” in the current system. 
One subject pilot, however, raised an interesting question about how often his aircraft would serve as a reference 
aircraft for another flight crew and therefore be ineligible to complete an ITP flight level change maneuver 
during actual oceanic flight operations, since “[e]veryone is always looking for a higher altitude.” The 
frequency of this situation occurring is a topic for further research. 

With respect to safety, several subject pilots voiced concerns about the possibility of “cheating” or “gaming” 
the system. For example, one subject pilot mentioned the potential for requesting a clearance with 15 nm 
separation with the intention of using a separation of 10 nm. Another subject pilot suggested that speed 
manipulations will occur, particularly when only “a couple of knots” are involved, and stated that flight crews 
may use speed changes to maintain separation as well as to “make a hole” through which their aircraft can 
maneuver. On the other hand, subject pilots expressed confidence in the ITP’s level of safety, especially since 
ATC would still have a presence in the system and be responsible for separation. A subject pilot suggested that 
potential increases in safety could be associated with the enhanced SA provided to the flight crew via the ITP 
display and stated that “no degradation of the system should take place” since ATC is still present in the system. 
When asked to comment on their use of free text phraseology to communicate with “ATC” during the 
experiment’s simulated flight scenarios, subject pilots indicated that, in general, they found that the “/L” and 
“/F” terminology to be straightforward and easy to learn and felt that “Leading” and “Following” were good 
terms to use since it is logical for pilots to reference their ITP Aircraft. It was pointed out, however, that airline 
companies often discourage the use of free text due to language barriers and the possibility of misinterpretation. 
Therefore, one subject pilot suggested that the ITP display provide “clickable” information that could be used to 
populate free text fields to reduce typographical errors and/or language differences (e.g., ITP climb/descend 
could be selectable). 

In addition to providing feedback regarding the ITP’s overall acceptability, safety, and phraseology, the 
subject pilots also shared enlightening viewpoints regarding ATC’s role in separation assurance during 
discussions that took place in the post-experiment group debrief sessions. Several of the subject pilots indicated 
that if the appropriate information is made available to flight crews, then the level of responsibility for 
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separation assurance that is given to pilots during ITP maneuvers is acceptable. One subject pilot stated that the 
level of pilot responsibility for separation assurance associated with the ITP is similar to that experienced while 
flying under Visual Metrological Conditions (VMC). However, another prevalent viewpoint was expressed 
when the potential for increasing pilot responsibility for separation assurance was considered. One commonly 
expressed opinion was that pilots do not want more responsibility for separation assurance, particularly because 
of “violation potential,” and that pilots have “no interest in being an air traffic controller.” One subject pilot 
stated that separation assurance is “ATC’s job,” and another subject pilot stated that responsibility for separation 
assurance should be kept “on the ground.” Yet another subject pilot stated that since flight crews don’t have 
access to the convergence information that ATC has that it would be a better use of resources to provide the 
additional information that makes the ITP possible to ATC rather than to flight crews. 

Subjective Assessments of the ITP’s Workload 

The subject pilots’ perceptions of the workload associated with performing standard flight level changes and 
ITP flight level changes were collected using the MCH Rating Scale, the post-experiment questionnaire, and 
feedback obtained during the post-experiment group debrief sessions. It should be noted that workload 
assessments are associated with a desktop PC-based operation and that the results are therefore relative to 
workload levels experienced during baseline operations for standard climbs and descents rather than being 
directly representative of workload levels experienced during actual flight operations. After completing each of 
the experiment’s 16 test conditions, subject pilots used the MCH Rating Scale to report the level of workload 
that they had just experienced. The post-experiment questionnaire was used to record the subject pilots’ 
descriptions of how the workload required to perform standard flight level changes during the experiment 
compared with the workload required to perform the ITP flight level changes. Feedback regarding their 
impressions of the workload associated with performing simulated flights East-to-West versus West-to-East 
over the Atlantic Ocean was obtained from the subject pilots during the post-experiment group debrief sessions. 

It was hypothesized that subject pilots would find the workload level associated with performing ITP flight 
level changes to be acceptable (i.e., that a subjective workload rating of “3” or less would be provided using the 
MCH Rating Scale) and that the subjective workload level experienced during ITP flight level changes would 
not be significantly higher than that experienced during standard flight level changes. As described below, 
nonparametric statistical tests were employed as a conservative method for analyzing workload ratings 
associated with the MCH Rating Scale’s discrete rating scale items. 

The MCH workload ratings associated with the requested standard scenarios revealed a mean of 1.20 
[Standard Deviation (SD) = 0.40, Sample Size (N) = 43], and the MCH workload ratings associated with the 
requested ITP scenarios revealed a mean of 1.58 (SD = 0.75, N = 149) (Figure 23). 
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Figure 23. Mean Modified Cooper-Harper (MCH) workload ratings associated with requested standard scenarios and 
requested ITP scenarios 


A Wilcoxon Test (a nonparametric within-subject test appropriate for analyzing two related samples of 
ordinal data) [13] was performed to evaluate the hypothesis that the workload level experienced by subject pilots 
during ITP flight level changes would not be significantly higher than that experienced during standard flight 
level changes. The results of this test failed to support the hypothesis since it revealed that a statistically 
significant difference existed between the workload ratings that subject pilots provided for the requested 
standard flight level change maneuvers as compared with the requested ITP flight level change maneuvers (p = 
0.0009). It is asserted, however, that the difference between the subject pilots’ workload ratings for the standard 
flight level change maneuvers and the ITP flight level change maneuvers is not operationally significant. 
Subject pilots indicated that other phases of flight (i.e.: approach) have much higher workload ratings than ITP 
flight level change maneuvers. This assertion is supported by the comments regarding workload that the subject 
pilots’ provided via the post-experiment questionnaire (see below). 


The mean MCH workload ratings associated with the eight types of requested flight level change maneuvers 
are shown in Figure 24. In this figure, means with different letters are significantly different in Wilcoxon Tests 
at p < 0.05 . 
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Figure 24. Mean Modified Cooper-Harper (MCH) workload ratings associated with requested flight level change 
maneuvers 


The subject pilots’ mean MCH workload ratings ranged from 1.42 to 1.95 for the requested ITP flight level 
change maneuvers (Figure 24). Therefore, these data support the hypothesis that subject pilots would find the 
workload level associated with performing ITP flight level changes to be acceptable (i.e., that a subjective 
workload rating of “3” or less would be provided using the MCH Rating Scale). 

A Friedman Test (a nonparametric within-subject test appropriate for analyzing three or more related 
samples of ordinal data) [12][13] was performed to determine if differences existed among the workload ratings 
that the subject pilots provided for the eight types of requested flight level change maneuvers. Since this test 
revealed that a significant difference existed between the workload ratings that subject pilots provided for at 
least two of the eight types of requested flight level change maneuvers (X 2 [7] = 44.2503; p = 0.0001), Wilcoxon 
Tests were performed to determine which of the requested flight level change maneuvers were perceived as 
having significantly different levels of workload. These Wilcoxon Tests revealed that the subject pilots 
provided significantly lower workload ratings for the requested standard climb maneuvers than they did for any 
of the requested ITP flight level change maneuvers (all p < 0.05) and that the subject pilots provided 
significantly lower workload ratings for the requested standard descent maneuvers than they did for the 
requested ITP combined climb flight level change maneuvers (p = 0.0325). 

Wilcoxon Tests were also performed to determine if differences existed between the workload ratings that 
subject pilots provided for requested flight level change maneuvers that: involved climbs versus descents; were 
achievable versus unachievable; and were associated with Eastbound versus Westbound flights. These tests 
revealed that significant differences did not exist between the workload ratings provided for requested flight 
level change maneuvers that: involved climbs (M = 1.53, SD = 0.77, N = 96) versus descents (M = 1.49, SD = 
0.71, N = 96) (p= 0.7389); were achievable (M = 1.49, SD = 0.65, N = 96) versus unachievable (M = 1.50, SD = 
0.75, N = 96) (p= 0.7630); or were associated with Eastbound (M = 1.54, SD = 0.78, N = 96) versus Westbound 
flights (M - 1.45, SD - 0.62, N = 96) (p= 0.3657). 

Through their post-experiment questionnaire responses, three subject pilots indicated that the workload level 
required to perform the ITP flight level changes was “about the same” as that required to perform the standard 
flight level changes, whereas the remaining nine subject pilots indicated that a “slightly higher” workload level 
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was required to perform the ITP flight level changes when compared to the standard flight level changes. The 
subject pilots who reported that the ITP flight level changes required a slightly higher workload level stated that 
the increase resulted from the additional key strokes required to submit the ITP flight level change request and 
the need to re-assess the situation to ensure that ITP distance and speed criteria were met prior to executing the 
flight level change maneuver. One subject pilot who stated that the workload level that he experienced was 
somewhat elevated for the performance of the ITP flight level change maneuvers as compared with the standard 
flight level change maneuvers also mentioned that he experienced an increase in workload when he performed 
ITP flight level change maneuvers involving two reference aircraft (i.e., combined climbs/descents) as compared 
with ITP flight level change maneuvers involving one reference aircraft (i.e., leading or following 
climbs/descents). Another subject pilot who reported experiencing a slightly higher level of workload when 
performing the ITP flight level change maneuvers as compared with the standard flight level change maneuvers 
stated that the increase in workload was offset by the improved SA provided by the ITP display. 

With respect to feedback obtained during the post-experiment group debrief sessions, subject pilots 
unanimously agreed that they experienced no difference in the level of workload required to complete simulated 
flight scenarios performed in an Eastbound versus Westbound direction. This finding supports the expectation 
that subject pilots would be capable of using the Oceanic Operations application’s vertical profile view (which 
depicted aircraft traveling from left-to-right across the lower portion of the EFB display screen) equally well 
during simulated flights performed East-to-West and West-to-East over the Atlantic Ocean. 

Supplemental Data Collection 

After completing the experiment’s 16 test conditions and the post-experiment questionnaire (but prior to the 
post-experiment group debrief session), the subject pilots were asked to perform an additional set of seven 
simulated flight scenarios as part of a supplemental data collection effort. The purpose of this effort was to 
assess the ability of the subject pilots to use the ITP to complete flight level change maneuvers involving more 
than one intervening flight level (i.e., to extend the use of the ITP to situations other than those involving the 
typical 2,000-foot flight level change that they had performed during the experiment’s simulated flight 
scenarios). While the experiment scenarios were identical for all subject pilots, the supplemental scenarios 
varied between subject pilots. Five of the supplemental scenarios were flown by all 12 subject pilots, and the 
remaining two supplemental scenarios were split evenly between the subject pilots. As a group, the subject 
pilots performed 72 supplemental ITP scenarios. However, due to a data collection error, selection error and 
execution error data were recorded for only 1 1 of the subject pilots resulting in the availability of these data for 
only 66 of the 72 supplemental ITP scenarios. 

Some of the supplemental ITP scenarios were designed such that multiple expected flight level change 
requests could be made. For example, during the “CC3_LF_UAD” scenario (Figure 25), the desired altitude 
was located 4,000 ft above the altitude at which the ITP aircraft was initialized in simulated flight. 
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Figure 25. Combined Climb through 3 Intervening Flight Levels 

Due to the spacing between the ITP aircraft and one of the intended reference aircraft (i.e., 13 nm), a combined 
climb maneuver was not possible when the scenario initially began, but became a viable maneuver 
approximately three minutes into the scenario. An alternative option available to the subject pilots during this 
scenario was to request two consecutive 2,000-foot ITP maneuvers, each with a single reference aircraft. The 
subject pilots could have considered this option as a possibility since an available altitude existed between the 
two potential reference aircraft. 

Since the supplemental ITP scenarios were not part of the formal experiment design matrix, statistical 
analyses were not performed using the data collected during the execution of these scenarios. However, 
expected versus requested scenario counts are provided; selection and execution errors are described; and the 
subject pilots’ workload ratings and comments regarding their impressions of the supplemental simulated flight 
scenarios are discussed. 

Selection and Execution Errors 

The subject pilots committed a total of eight selection errors while performing the supplemental ITP 
scenarios. One of the supplemental ITP scenarios was designed such that one of the two expected requests that 
subject pilots could make involved a standard climb maneuver. Since seven of the subject pilots requested the 
standard climb maneuver instead of making the available ITP request, seven of the 66 expected supplemental 
ITP scenarios were “converted into” requested supplemental standard scenarios. Therefore, the supplemental 
data collection effort involved seven requested standard scenarios and 59 requested ITP scenarios. 

While execution errors were not committed during the requested standard scenarios, nine execution errors 
occurred during eight of the 59 requested ITP scenarios. One of the execution errors was classified as a safety- 
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related execution error since the subject pilot failed to maintain vertical speed while executing a flight level 
change maneuver. The remaining eight execution errors were classified as being non-safety-related execution 
errors and were similar to those that occurred during the experiment’s 16 test conditions. Six execution errors 
occurred when subject pilots requested an incorrect altitude during a flight level change request; one execution 
error occurred when a subject pilot referenced an aircraft at the desired altitude; and one execution error 
occurred when a subject pilot made a typographical error in the ITP syntax used to make a flight level change 
request. 

It is important to realize that the supplemental data are not directly comparable to the data collected during 
the experiment’s 16 test conditions for a number of reasons. Although subject pilots were trained to use the ITP 
to execute flight level change maneuvers through multiple intervening flight levels, they only had the 
opportunity to practice and perform 2,000-foot ITP maneuvers during the pre-experiment training session as 
well as throughout the experiment itself. Also, the 2,000-foot ITP maneuvers that subject pilots completed prior 
to the supplemental data collection session were intermingled with scenarios involving standard flight level 
changes, and the experiment’s 16 test conditions were presented in random order. Hence, the results of the data 
obtained during the experiment cannot be compared directly to the supplemental data due to possible sequence 
effects. However, it is still interesting to loosely compare the numbers and types of execution errors that 
occurred during the experiment’s 149 requested ITP scenarios with those that occurred during the supplemental 
data collection session’s 59 requested ITP scenarios. 

Subject pilots committed at least one execution error during 28 of the experiment’s 149 requested ITP 
scenarios and committed at least one execution error during eight of the supplemental data collection session’s 
59 requested ITP scenarios. Therefore, execution errors occurred in 18.8% of the experiment’s scenarios 
involving ITP flight level change requests and in 13.6% of the supplemental data collection session’s scenarios 
involving ITP flight level change requests. Since nine of the experiment’s 149 requested ITP scenarios involved 
safety-related execution errors, subject pilots committed safety-related ITP execution errors 6.0% of the time. In 
comparison, one of the supplemental data collection session’s 59 requested ITP scenarios involved a safety- 
related execution error, therefore subject pilots committed safety-related ITP execution errors 1.7% of the time. 

Subjective Assessments of Workload 

After completing each of the supplemental data collection session’s simulated flight scenarios, subject pilots 
used the MCH Rating Scale to report the level of workload that they had just experienced. As shown by the 
mean MCH workload ratings presented in Figure 26, the subject pilots’ mean MCH workload ratings ranged 
from 1.42 to 2. These data appear to indicate that subject pilots found the workload level associated with 
performing the supplemental data collection session’s scenarios to be acceptable. 
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Figure 26. Mean Modified Cooper-Harper (MCH) workload ratings associated with the supplemental data collection 
session’s scenarios (for scenario description see Table 17) 

Table 17. Supplement data collection scenario description 


# 

Scenario name 

Scenario description 

1 

2 

3 

4 

5 

6 
7 

CC2FFAC 
CC2 LF AC 
CC2 LL AC 
CC3LFAC 
CC3LFUAD 
CC3POSTEXP 
FC3 UAT 

Combined Following/Following Climb through 2 Flight Levels - Accept 
Combined Leading/Following Climb through 2 Flight Levels - Accept 
Combined Leading/Leading Climb through 2 Flight Levels - Accept 
Combined Leading/Following Climb through 3 Flight Levels - Accept 
Combined Leading/Following Climb through 3 Flight Levels - Unable distance 
Combined Leading/Following Climb through 3 Flight Levels - Accept 
Following Climb through 3 Flight Levels - Unable traffic 


Subjective Assessments of the Supplemental Data Collection Session ’s Scenarios 

With respect to feedback obtained during the post-experiment group debrief sessions that took place 
following the supplemental data collection sessions, subject pilots indicated that they were able to use the ITP to 
perform 3,000 and 4,000-foot flight level change maneuvers during the supplemental data collection session’s 
scenarios just as they had used it to perform 2,000-foot flight level change maneuvers during the experiment’s 
scenarios. In general, the subject pilots did not believe that safety would be compromised by using the ITP to 
maneuver through multiple intervening flight levels. However, it was suggested by one subject pilot that 
altitude change requests involving only a single intervening flight level might help him avoid climbing or 
descending to an incorrect altitude. In terms of the usefulness of making altitude changes through multiple 
intervening flight levels, several of the subject pilots stated that flight crews typically do not request 3,000 or 
4,000-foot climbs or descents due to aircraft performance limitations, indicating that such flight level changes 
would not be useful. 
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Summary & Conclusions 


Under sponsorship of NASA LaRC’s EOO research element, a HITL flight simulation experiment was 
conducted in September 2006 to assess the validity and pilot acceptability of a proposed new air traffic “In Trail 
Procedure” or ITP. The ITP operational concept was developed by researchers at NASA LaRC and others in the 
international ATM research community, and if deployed would represent an early application of airborne ADS- 
B technology that would increase opportunities for ITP-equipped aircraft to fly at their optimal altitudes on 
flights in oceanic and remote non-radar flight regions. This experiment represented a first opportunity to present 
airline pilots with the ITP in a controlled experimental environment, with a medium-fidelity set of simulated 
operational scenarios and an initial prototype Oceanic Operations application, and to evaluate their ITP 
performance and impressions. 

In-house preparations for this experiment included developing a prototype ITP Oceanic Operations software 
application and simulating an EFB hardware device on which this application was displayed. Preparations also 
included extensive modifications to NASA LaRC’s ATOL simulation software, including the development of 
new air traffic modules to simulate appropriate air traffic controller responses and the generation of realistic air 
traffic simulation scenarios on the NATOTS. Many such scenarios were generated in support of an experiment 
matrix designed to systematically explore pilot responses to 16 situations where different types of ITP request 
geometries and maneuvers were, or were not, applicable. An additional 7 scenarios were also developed to 
explore pilot responses to more complex ITP situations after completion of the experiment’s 16 test conditions. 

Twelve commercial airline pilots, all with current oceanic flight experience, participated in the experiment in 
groups of up to four, with each group participating in the study over the course of two consecutive days. The 
subject pilots were provided with training and practice sessions regarding ITP concepts and procedural steps, as 
well as on the operation of relevant aircraft simulation equipment in the ATOL. The subject pilots then flew the 
simulated ITP scenarios and answered questionnaires at the end of each scenario, as well as answered a final 
questionnaire and participated in a group discussion at the conclusion of all scenarios. The subject pilots’ 
subjective assessments of ITP validity and acceptability were measured via these questionnaires and discussion 
results, and their objective performance in appropriately selecting, requesting, and performing ITP flight level 
changes, where ITP flight level changes might be appropriate, was evaluated for each scenario. Objective 
performance and subjective workload assessment data from the experiment’s 16 test conditions were analyzed 
for statistical and operational significance and discussed. 

In the majority of scenarios, subject pilots were able to correctly assess the traffic situation, select an 
appropriate response (i.e., either a standard flight level change request, an ITP request, or no request), and 
execute their selected flight level change procedure, if any, without error. Subjective workload assessments 
were slightly higher for ITP than for standard flight level change requests, but were well within acceptable 
limits. In addition, the subject pilots’ subjective acceptability ratings and comments regarding the ITP were 
generally high and positive, respectively. Based on these results, the ITP is asserted to be generally both valid 
and acceptable for the experiment scenarios as flown in a medium fidelity simulation environment. 

Regarding the validity of the procedure, it was hypothesized that subject pilots would not find any missing, 
incomplete, or extraneous procedural steps associated with the ITP, but at least one subject pilot stated that 
additional steps should be included to “avoid syntax errors” and provide an additional “independent check” (i.e., 
crew crosscheck procedures). During the discussion, potential concerns were raised with regard to the existence 
of wake vortices during the execution of an ITP maneuver, as well as the impact of non-participating aircraft on 
the success or failure of the ITP. Subject pilots also mentioned that an emphasis will need to be placed on 
constant Mach operations since current day operations in oceanic regions outside of the North Atlantic heavily 
rely on the FMC’s “econ” setting, which is not a constant Mach function. 
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All 12 subject pilots indicated that the ITP was an acceptable procedure, but two subject pilots mentioned 
concerns regarding the procedure’s reliance on an “honor system” in which the distances among aircraft must be 
accurately reported to ATC, and several subject pilots voiced concerns about the possibility of “cheating” or 
“gaming” the system. Some subject pilots indicated that the ITP’s acceptability would increase if pilots were 
afforded the opportunity to communicate alternate altitude requests to ATC and if the requirement for constant 
Mach was eliminated. 

From comments regarding potential benefits that the ITP might have for flight crews, passengers, and airline 
companies, subject pilots identified potential benefits for flight crews as being: improved traffic SA; more 
efficient aircraft operation; enhanced job satisfaction associated with the opportunity to achieve optimum 
performance; and improvements in safety due to the avoidance of weather and turbulence. For passengers, the 
subject pilots identified potential benefits as being: smoother rides due to the flight crew’s ability to vacate 
turbulent altitudes; on-time arrivals since delays due to oceanic congestions may be avoided and improved ticket 
prices if airline companies experience savings. For airline companies, the subject pilots identified: fuel savings; 
safety and passenger comfort; and improved scheduling due to route planning as potential benefits. 

While the ITP was generally regarded to be valid and acceptable, the error rates for ITP maneuvers were 
higher than for standard flight level changes, and these errors have design implications for both the ITP and the 
Oceanic Operations application prototype traffic display. It is beyond the scope of this paper to evaluate the 
operational acceptability of the reported error rates, but several explanations and mitigation strategies for the 
various errors are discussed in the paper. In summary, these error mitigation strategies might involve: 1) 
improving the prototype Oceanic Operations display interface and symbology; 2) limiting the applicability of 
the ITP to well-defined simple geometries; 3) defining specific rules-of-thumb (versus the current judgment- 
based guidance) as to when pilots should request an ITP; and/or 4) increasing the level of automation/decision 
support provided by the Oceanic Operations application. 

In summary, the subject pilots found the procedure valid, acceptable, “procedurally sound,” and “intuitively 
obvious” and were very enthusiastic about the enhanced SA provided by the Oceanic Operations application 
display. The subject pilots were less excited about more future pilot responsibility for separation assurance, but 
would like to see the ITP implemented for use during actual oceanic flight operations and were interested in 
knowing how quickly this might happen. 
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Appendix A 

Modified Cooper-Harper Rating Scale Introductory Materials 

Overview 


After completing each scenario, you will be asked to give a rating on a Modified Cooper-Harper Scale for 
workload. This rating scale and important definitions are described below. 

Important Definitions 

To understand and use the Modified Cooper-Harper Scale properly, it is important that you understand the terms 
used on the scale and how they apply in the context of this experiment. 

First, the “instructed task” is the flight task you have been assigned to perform in this experiment. It includes 
flying the aircraft simulator within specified levels of accuracy and performing all duties that are requested of 
you during the time interval designated by the experimenters. 

Second, the “operator” in this situation is you. Since the scale can be used in different situations, the person 
performing the ratings is called an operator. You will be operating the system and then using the rating scale to 
quantify your experience. 

Third, the “system” is the complete group of equipment you will be using in performing the instructed task. 
Together you and the system make up the “operator/system.” 

Fourth, “errors” include any of the following: mistakes, incorrect or incomplete actions or responses, and 
blunders. In other words, errors are any appreciable deviation from desired “operator/system” performance. 

Finally, “mental workload” is the integrated mental effort required to perform the instructed task. It includes 
such factors as level of attention, depth of thinking, and level of concentration required by the instructed task. 

Rating Scale Steps 

On the Modified Cooper-Harper Scale, you will notice that there is a series of decisions that follow a 
predetermined logical sequence. This logic sequence is designed to help you make more consistent and accurate 
ratings. Thus, you should follow the logic sequence on the scale for each of your ratings in the experiment. 

The steps that you will follow in using the rating scale logic are as follows: 

1 . First, you will decide if the instructed task can be accomplished most of the time; if not, then your rating 
is a 10, and you should circle 10 on the rating scale. 

2. Second, you will decide if adequate performance is attainable. Adequate performance means that the 
errors are small and inconsequential in performing the instructed task. If errors are not small and 
inconsequential, then there are major deficiencies in the system, and you should proceed to the right. 

By reading the descriptions associated with the numbers 7, 8, and 9, you should be able to select the one 
that best describes the situation you have experienced. You would then circle the most appropriate 
number. 

3. If adequate performance is attainable, your next decision is whether or not your mental workload for the 
instructed task is acceptable. If it is not acceptable, you should select a rating of 4, 5, or 6. One of these 
three ratings should describe the situation you have experienced, and you would circle the most 
appropriate number. 
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4. If mental workload is acceptable, you should move to one of the top three descriptions on the scale. 

You would read and carefully select the rating 1, 2, or 3 based on the corresponding description that best 
describes the situation you have experienced. You would circle the most appropriate number. 

Remember that you are to circle only one number, and the number should be arrived at by following the logic of 
the scale. You should always begin at the lower left and follow the logic path until you have decided on a 
rating. In particular, do not skip any steps in the logic. Otherwise, your rating may not be valid and reliable. 

How You Should Think of the Rating 

Before you begin making ratings, there are several points that need to be emphasized. First, be sure to try to 
perform the instructed task as instructed and make all of your evaluations within the context of the instructed 
task. Try to maintain adequate performance as specified for your task. 

Second, the rating scale is not a test of your personal skill. On all of your ratings, you will be evaluating the 
system for a general user population, not yourself. You may assume that you are an experienced member of that 
population. You should make the assumption that problems you encounter are not problems you created. They 
are problems created by the system and the instructed task. In other words, don’t blame yourself if the system is 
deficient; blame the system. 

Third, try to avoid the problem of nit picking an especially good system, and of saying that a system that is 
difficult to use is not difficult to use at all. These problems can result in similar ratings for systems with quite 
different characteristics. Also, try not to overreact to small changes in the system. This can result in ratings that 
are extremely different when the systems themselves are quite similar. Thus, to avoid any problems, just always 
try to “tell it like it is” when making your ratings. 
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The Rating Scale 


Difficulty Level 



Operator Demand Level 


Rating 


VERY EASY, 
Highly desirable 

Operator mental effort is minimal and 
desired performance is easily attainable 

1 

EASY, 

Desirable 

Operator mental effort is low and 
desired performance is attainable 

2 

FAIR, 

Mild Difficulty 

Acceptable operator mental effort is required 
to attain adequate system performance 

3 


MINOR BUT ANNOYING 
DIFFICULTY 

Moderately high operator mental effort is required 
To attain adequate system performance 

4 

MODERATELY OBJECTIONABLE 
DIFFICULTY 

High operator mental effort is required 
To attain adequate system performance 

5 

VERY OBJECTIONABLE BUT 
TOLERABLE DIFFICULTY 

Maximum operator mental effort is required 
To attain adequate system performance 

6 


IMPOSSIBLE 


Instructed task cannot be accomplished reliably 


MAJOR DIFFICULTY 

Maximum operator mental effort is required 
To bring errors to moderate level 

7 

MAJOR DIFFICULTY 

Maximum operator mental effort is required 
To avoid large or numerous errors 

8 

MAJOR DIFFICULTY 

Intense operator mental effort is required 
To accomplish task, but frequent or 
Numerous errors persist 

9 


10 
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Appendix B 

ITP Display Interface User Survey 


Thank you! 

Thank you for participating in the In-Trail Procedure (ITP) Evaluator Display User Survey. 

Background 

NASA and the FAA are developing an airborne surveillance system which allows a properly equipped aircraft to 
determine and display the position, altitude, groundspeed, direction of flight, and the identification of aircraft in 
surrounding airspace. This surveillance system is based on a technology known as Automatic Dependent 
Surveillance-Broadcast (ADS-B), which transmits aircraft position and other essential data via the radar 
transponder. Aircraft equipped to receive ADS-B data can then determine the position, ground track and altitude 
of other ADS-B equipped aircraft. With aircraft properly equipped, ADS-B can serve as a precise surveillance 
system, ultimately enabling new operational procedures. ADS-B will have an operational range of 
approximately 150NM, and is expected to be especially useful in non-radar environments. 

Purpose of the Survey 

NASA is developing an In-Trail Procedure (ITP) that is based on the ADS-B. Using this ADS-B enabled 
procedure, flight crews could request and receive clearance to change altitudes more often while operating in 
non-radar environments. This research is being conducted in coordination with the aviation industry through the 
RTCA/EUROCAE Requirements Focus Group, reference ADS-B Initiated Flight Level Change in Oceanic and 
Remote Airspace (ATSA-AIFLC) OSED version 3.6. The purpose of this survey is to collect your ideas and 
opinions regarding the design of an ITP display interface. The ITP potentially could be operated on an 
Electronic Flight Bag (EFB) independently of other flight guidance and navigation displays. The information 
contained in this document will assist pilots in assessing the feasibility of requesting an ITP climb or descent in 
oceanic airspace. Your feedback will help us define the display’s requirements so that when this becomes an 
operational system flight crews are ultimately provided with a useful and usable tool. Input from subject matter 
experts and line pilots who will be the eventual “end users” of the ITP Evaluator is critical to the successful 
development of this tool, hence your participation is very much appreciated. 

Executive Summary of the ITP 

Aircraft in oceanic and remote non-radar airspace frequently fly for extended periods of time in the same 
direction and along the same or similar paths over the earth’s surface as other aircraft. Oceanic controllers utilize 
procedural separation to ensure aircraft remain separated while flying in this airspace. Particularly during such 
long flights, these aircraft may enhance operational efficiency and safety through a change of flight level by 
climbing (usually to increase fuel efficiency) or descending (usually to avoid turbulence or unfavorable winds). 
In many situations however, the standard longitudinal separation does not exist at the next higher or lower flight 
level, but does exist at the succeeding flight level. The aircraft desiring the flight level change would therefore 
be prevented or “blocked” from making the climb or descent. Leveraging the capabilities of ADS-B and 
onboard automation, the ITP concept has been designed to enable the use of temporarily reduced distance or 
time -based separation minima during the maneuver to allow an aircraft to pass through the intervening flight 
level of the blocking aircraft. To make this possible, an onboard ITP Evaluator is being designed. The ITP 
Evaluator will reduce pilot workload by gathering all relevant information from the surrounding airspace 
(approximately 150 nm radius), computing the suitability of the ITP maneuver, and displaying that suitability 
status to the flight crew. The crew would then decide to request an ITP clearance to make an altitude change, 
providing the controller with appropriate call sign and separation information from the ITP Evaluator using a 
new standard phraseology. This is a crucial step, as there are additional separation and spacing criteria beyond 
the scope of the ITP Evaluator that the controller must consider before issuing a clearance. Additional 
information regarding the ITP maneuvers and the ITP Evaluator is available in the detailed description sections. 


Survey Instructions 

Please take a few moments to review and understand the “Introduction to the ITP Maneuvers” and “Introduction 
to the ITP Evaluator” before proceeding to the survey questions section. Once you have completed the survey, 
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please return your responses to the NASA Langley Research Center using the postage paid envelope that has 
been provided. Please keep the introductory material for your records. 

Please note that the information collected from individual survey respondents will be kept strictly confidential. 
At no time will your individual responses be released to anyone other than individuals working directly on the 
project without your written consent. The information you provide will have your name removed, and only a 
participant number (i.e., Participant #1, Participant #2, etc.) will identify your responses during analyses and any 
written reports regarding the results of this survey. 

If you have any questions about this survey now or in the future, please contact: 

Jennifer L. Murdoch 

Aircraft Operations & Evaluation Branch 

Mail Stop 156A, NASA Langley Research Center, Hampton, VA 23681-2199 

Phone: (757) 864-8304 

Email: Jennifer.L.Murdoch@nasa.gov 

Introduction to the ITP Maneuvers 

The ITP enables an aircraft to perform a climb or descent maneuver to a requested flight level through one 
intervening flight level that is occupied by another aircraft (referred to as a potentially blocking aircraft) on the 
same oceanic track. A total of six ITP climb and descent maneuvers are defined: following climbs and descents, 
in which the ownship is behind the other aircraft; leading climbs and descents, in which the ownship is ahead of 
the other aircraft; and combined climbs and descents, in which the ownship is behind one aircraft at the 
intervening flight level, but ahead of another. A typical ITP climb initial condition scenario, on a same-direction 
track structure in RVSM airspace, is presented in the figure below. In this example, the aircraft at FL 350 is 
defined as the reference aircraft. 

Each ITP maneuver terminates when the ownship reports established at the new flight level. Any additional 
flight level changes would be initiated via either a new ITP maneuver request, if a potentially blocking aircraft is 
present, or otherwise a regular non-ITP flight level change request. 

The ITP requires the flight crew to use information derived on the flight deck to determine if the criteria for an 
ITP are met, and to relay some of this information to ATC for a clearance decision. The following aircraft and 
flight crew requirements must be met before requesting an ITP: 

• The ownship aircraft must have ADS-B in equipment that provides the flight crew with the flight 
identifier, altitude, range (from the ownship to the reference aircraft), and ground speed differential of 
the potentially blocking ADS-B out equipped aircraft. 

• The flight crew must be properly qualified for ITP maneuvers. 

• The airline operational specifications for the ownship aircraft must permit use of the ITP. 

• The ownship aircraft must be able to maintain cruise Mach throughout the procedure. 

• The ownship aircraft must also be able to maintain a minimum climb or descent rate of 300 fpm until 
established at the requested flight level. 

• The ownship ’s position data must meet the ITP accuracy requirements, and the ADS-B data from the 
potentially blocking aircraft must be qualified for meeting the requirements of the ITP. 

In addition to the requirements listed above, specific criteria for range and ground speed differential with 
potentially blocking aircraft must be met in order to request an ITP maneuver. When the range from ownship to 
the other aircraft is less than 15 nm, the ITP is not applicable. When the range is between 15 and 20 nm, the 
ground speed differential must not exceed 20 kt of closure. When the range is greater than 20 nm, the ground 
speed differential must not exceed 30 kt of closure. These range and ground speed differential criteria are 
applicable to all six ITP climb or descent maneuvers. 
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FL 360 



Typical initial condition for executing an ITP climb. 


The sequence followed by the ownship flight crew and the oceanic controller is as follows: 

1. The ownship flight crew decides that a flight level change is desirable based on fuel efficiency, ride 

quality, or other considerations. The crew also determines, based on the evaluator display, that an ITP 

maneuver appears to be required. 

2. The ownship flight crew reviews the ITP Evaluator information to determine that the criteria for executing 
an ITP maneuver are met. These criteria include crew qualifications, aircraft equipage and performance 
capabilities, and the relative ranges and groundspeed differentials with all potentially blocking aircraft at 
intervening flight levels. 

3. The ownship flight crew requests clearance from ATC to conduct the appropriate ITP maneuver to the 
requested flight level. The request will include the flight identifiers of any reference aircraft (i.e., 
potentially blocking aircraft that meet the ITP criteria) and the range in nautical miles to each of those 
reference aircraft. This request will be made using a new standard phraseology. 

4. The controller will assess the requested altitude using standard separation minima and procedures, and 

will grant the request if all applicable criteria are met. These criteria include standard longitudinal 

separation minima, limits on the acceptable Mach difference between the ownship and reference aircraft 
(maximum of 0.03 Mach), knowledge of the positions and speeds of all other ADS-B equipped and non- 
ADS-B equipped aircraft that may be in the area, and any other appropriate considerations. Alternatively, 
the controller may, if appropriate, issue a standard climb clearance. At all times, the controller maintains 
responsibility for separation, granting clearance for the maneuver based on the information provided by 
the requesting aircraft. The controller also ensures that the reference aircraft does not maneuver during the 
ITP maneuver. 

5. After receiving the clearance to conduct the ITP, the ownship flight crew again verifies that the ITP 
criteria are met immediately before initiating the ITP climb or descent maneuver, and initiates the 
maneuver without delay. However, after acceptance of the ITP clearance and initiation of the maneuver, 
the ownship flight crew is not required to monitor any information regarding the reference aircraft, such as 
range or ground speed difference - this information was only necessary to make the ITP request. 

6. While performing the ITP maneuver, the ownship flight crew monitors their speed as well as their climb 
or descent rate. If the maneuver cannot be accomplished at the original cruise Mach and also meet the 
minimum continuous climb or descent rate of 300 fpm, the ownship aircraft must return to its initial flight 
level, and the flight crew must immediately notify the controller. 

7. Upon issuance of the ITP clearance, the controller will protect both the initial flight level and the 
requested flight level until the controller receives a report from the requesting aircraft that it is established 
at the requested flight level (or that it has returned to the initial altitude, in case it cannot complete the 
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ITP). The controller will also refrain from issuing any maneuver clearances to the reference aircraft until 
the requesting aircraft reports that it has completed maneuvering. 

8. The ownship flight crew reports to the controller when they have reached the requested flight level. This 
completes the ITP. 


Introduction to the ITP Evaluator 

The ITP Evaluator is intended to serve as a tool that flight crews may interact with via a cockpit mounted EFB. 
These notional depictions of the information required to perform an ITP are not intended to be used as 
navigation displays. The ITP Evaluator tracks, analyzes, and displays relevant same-track ADS-B air traffic in 
the ITP range according to the rules described in the Introduction to the ITP Maneuvers section, and operates 
independently of other flight guidance, navigation and traffic displays. This section presents several candidate 
EFB presentation configurations. Definitions of some key words, display functionality, candidate display 
combinations, and additional features are provided. 

Definitions: 

Vertical : Two dimensional view of airspace providing multiple altitudes with no graphical lateral or track 
information. 

Horizontal : Two dimensional view of airspace providing look-down perspective with no graphical altitude 
information. 

Track Up : Presentation of path upward on the Horizontal view. 

Track Right : Presentation of path towards the right on the Horizontal view. 

Display Functionality: 

On all displays, the ownship aircraft is represented by a solid white triangle, and “same-track” traffic aircraft are 
depicted by hollow cyan triangles on the vertical display and by hollow cyan chevrons on the horizontal display. 
An aircraft is considered to be “same-track” if its lateral distance and flight path angle fall within standards- 
defined limits with respect to the ownship aircraft. The apex of the triangles and chevrons indicates the direction 
of travel of the depicted aircraft, as well as its center of mass. A cyan data tag containing flight identifier, range, 
and groundspeed differential (difference between ownship and reference aircraft) is provided for potentially 
blocking aircraft located at intervening flight levels, and a cyan “unavailable” tag is used to indicate that a 
blocking aircraft is preventing an ITP maneuver through its respective flight level. The groundspeed differential 
is presented with a triangle-arrow that either points towards or away from the ownship. The arrow points 
towards the ownship when the distance between the ownship and the tagged aircraft is decreasing while the 
arrow points away from the ownship when the distance is increasing. On the horizontal displays, the data tag 
follows TCAS conventions. The ownship MCP altitude is depicted on the vertical display as a solid magenta 
line, while other flight levels are depicted as dotted lines. These examples depict a non-organized track system 
region such as the South Pacific. 

Vertical and horizontal display range selectors provide an interface for choosing the desired visible ranges to 
display. A display range selector is located at the bottom left corner of the display and indicates the current total 
visible display range. It is associated with buttons that allow the user to increase or decrease the visible display 
range. 

An altitude range selector is located at the bottom center of the display and indicates the current total visible 
display altitude range. It is associated with buttons that allow the user to increase or decrease the visible altitude 
display range. Note, however, that display ranges and altitudes in excess of those usable by the ITP can be 
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presented. These are provided as reference ranges and altitudes only to facilitate situation awareness. 
Information regarding accessibility at these greater altitude ranges is not presented. 

Candidate Display Configurations: 

Figure 1 presents the “Vertical-toggle-Horizontal” configuration (described below) with annotations that explain 
the encoding details of the symbology. Figures 2-4 present the configurations as they might actually appear on 
an EFB. 

Vertical-toggle-Horizontal Track Up : This candidate is presented in Figure 2a (and also in Figure 1 with 
annotations). This configuration implements each of the two views as separate pages on the EFB, requiring the 
user to manually switch or “toggle” between them. The Horizontal view is presented as Track Up. It is 
envisioned that an EFB control device (e.g., bezel button, rotary knob, switch, etc.) will be used to toggle 
between vertical and horizontal ITP Evaluator display formats. 

Vertical-toggle-Horizontal Track Right : This candidate is presented in Figure 2b, and is the same as Figure 2a, 
except that the Horizontal view is presented with the aircraft tracking to the right. 

Combined with Vertical-on-Bottom, Track Up : This candidate is presented in Figure 3a. This configuration 
combines each of the two views on a single page on the EFB, with the Vertical view below the Horizontal view. 
The Horizontal view is presented as Track Up. 

Combined with Vertical-on-Bottom, Track Right : This candidate is presented in Figure 3b, and is the same as 
Figure 3a, except that the Horizontal view is presented with the aircraft tracking to the right. 

Combined with Vertical-on-Top, Track Up : This candidate is presented in Figure 4a. This configuration 
combines each of the two views on a single page on the EFB, with the Vertical view above the Horizontal view. 
The Horizontal view is presented as Track Up. 

Combined with Vertical-on-Top, Track Right : This candidate is presented in Figure 4b, and is the same as 
Figure 4a, except that the Horizontal view is presented with the aircraft tracking to the right. 
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Figure 1. Vertical-toggle-Horizontal, Track Up Candidate EFB configuration, with annotations 


65 



SHOW 

HORIZ 


SHOW 

VERT 


370 -C^ T , r f f 

UAL 123 

360 r T T 

350 1 K 1 1 ^ 

80 40 0 40 80 

340 r T DAL789 ■■" T 1 1 

19NM 12KT^ 

330 ^ — r — ™ 1 


\ & 


x 




LO 




A 

-020 


, fc. 

+0fl?K 

B0 + 


40 + 


UAU123 

+010 


UNAVAILABLE 



/ 

^ / 






co 

co 


X 


\ 


200 NM 


ZOOM 

IN 


ZOOM 

OUT 


4000 FT 


+■ ALT - ALT 

RANGE RANGE 


Figure 2a. Vertical-toggle-Horizontal, Track Up Candidate EFB configuration 
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Figure 2b. Vertical-toggle-Horizontal, Track Right Candidate EFB configuration 
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Figure 3a-b. Combined with Vertical-on-Bottom, (a) Track Up and (b) Track Right Candidate EFB configurations 
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Figure 4a-b. Combined with Vertical-on-Top, (a) Track Up and (b) Track Right Candidate EFB configurations 
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In-Trail Procedure (ITP) Evaluator Display User Survey: Questionnaire 


Now that you have reviewed the ITP Evaluator Display User Survey’s introductory materials and have 
acquired an understanding of the proposed ITP maneuvers and the ITP Evaluator, you are now prepared to 
complete the survey questions section. The primary aim of the survey questions is to elicit your feedback 
regarding the ITP Evaluator display interface. Additional demographic questions are also included so that 
general characteristics of the survey participants may be understood. 

With respect to the ITP Evaluator display interface, you will be asked to comment on the: 

• three proposed display formats (i.e., Vertical-toggle-Horizontal; Combined with Vertical-on- 
Bottom; and Combined with Vertical-on-Top); 

• Horizontal Track Up configuration versus the Horizontal Track Right configuration; and 

• use of symbology, color, and information coding. 

Please refer to the figures included in the introductory materials while you complete the questionnaire. 

Throughout the survey, spaces are provided for you to explain your selections as well as suggest 
alternatives and/or improvements. Please provide as much or as little detail in your written responses as you 
would like, and feel free to use the back of each sheet to continue your comments if needed. Additionally, 
you will have an opportunity to illustrate your ideal ITP Evaluator display interface. 

With respect to demographic information, you will be asked to provide information regarding your: 

• military, business/corporate, and/or scheduled airline experience 

• general flying experience; 

• experience with Vertical Situation Displays (VSDs), Electronic Flight Bags (EFBs), and Personal 
Computers (PCs); and 

• gender, date of birth, willingness to be contacted by researchers to discuss your survey responses, 
and interest in participating in future ITP simulation experiments. 

Please complete and return this survey within one week of receiving it to the NASA Langley Research 
Center using the postage paid envelope that has been provided. Thank you for participating in the ITP 
Evaluator Display User Survey. 
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Display Interface Feedback 


1. Various combinations of the vertical and horizontal graphical displays have been presented. Please rank 
the display formats listed below according to your preferences. 

The rank of 1 should correspond with the display format you like the most. 

The rank of 6 should correspond with the display format you like the least. 

Vertical-toggle-Horizontal, Track Up, Figure 2a 

Vertical-toggle-Horizontal, Track Right, Figure 2b 

Combined with Vertical-on-Bottom, Track Up, Figure 3a 

Combined with Vertical-on-Bottom, Track Right, Figure 3b 

Combined with Vertical-on-Top, Track Up, Figure 4a 

Combined with Vertical-on-Top, Track Right, Figure 4b 

Please explain your rank ordering: 


2. Considering the display format you ranked as number 1 (i.e., liked the most): 

a. How useful do you think this display format would be for supporting the six ITP maneuvers? 
(Circle one response.) 

Very Useful Somewhat Marginally Not at all 

Useful Useful Useful Useful 

b. Do you have a suggested alternative or improvement, particularly if you think that this display 
format would not be useful for supporting a particular procedure? 
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c. If you do not think you could safely execute the full range of ITPs using this display format, which 
procedure(s) do you think would be difficult to perform? Why? 


d. What information could be added to, removed from, or changed in this display format to increase 
the safety of executing ITPs or otherwise improve the display interface? 


2.1. If you wish, you may comment on a second display format (and any others as well); otherwise, skip to 
question 3 on page 4: 

a. What ranking number are these comments about? 

b. How useful do you think this display format would be for supporting the new ITP maneuvers? 

Very Useful Somewhat Marginally Not at all 

Useful Useful Useful Useful 

c. Do you have a suggested alternative or improvement, particularly if you think that this display 
format would not be useful for supporting a particular procedure? 
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d. If you do not think you could safely execute the full range of ITPs using this display format, which 
procedure(s) do you think would be difficult to perform? Why? 


e. What information could be added to, removed from, or changed in this display format to enhance 
your ability to perform an ITP, or otherwise improve the display interface? 


3. Do you feel that the inclusion of the Horizontal display format enhances the ITP Evaluator? 

Yes No 

Please explain your selection: 


4. Which Horizontal display orientation do you prefer? 
Track Up Track Right 

Please explain your selection: 
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a. What information could be added to, removed from, or changed in the Horizontal Track Up display 
format to increase the safety of executing ITP maneuvers or otherwise improve the display interface? 


5. How useful would it be to have the ability to simplify the display interface by requesting that either a 
“climb only” or a “descent only” option be depicted? A climb only configuration would show only those 
flight levels above the ownship on the vertical display, and for both displays, only those aircraft at higher 
flight levels. Conversely, a descent only option would show only lower flight levels and aircraft. 


Very Useful Somewhat Marginally Not at all 

Useful Useful Useful Useful 

Please explain your selection: 


6. Consider the display interface’s symbology: 


a. How appropriate is the use of color? 

Very Appropriate Somewhat Neutral Somewhat Inappropriate Very 

Appropriate Appropriate Inappropriate Inappropriate 

b. Do you have suggested alternatives or improvements, particularly if you found the display 
interface’s use of color to be inappropriate? 
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c. How appropriate is the information included in the traffic aircraft’s data tags? 

Very Appropriate Somewhat Neutral Somewhat Inappropriate Very 

Appropriate Appropriate Inappropriate Inappropriate 

d. Do you have suggested alternatives or improvements, particularly if you found the information 
included in the traffic aircraft’s data tags to be inappropriate? 


e. Is it helpful to have the word “unavailable” included in a traffic aircraft’s data tag when applicable 
(see examples below)? 

Yes No 



Please explain your selection: 


f. How intuitive is the representation of the ground speed differential between the ownship and the 
potentially blocking aircraft (i.e., the use of an arrow pointing to the ownship when the distance 
between the ownship and the tagged aircraft is decreasing, and with the arrow pointing away from 
the ownship when the distance is increasing)? 

Very Intuitive Somewhat Neutral Somewhat Unintuitive Very 

Intuitive Intuitive Unintuitive Unintuitive 
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Demographic Information 

General Experience 

1 . Please provide your best estimate for each of the following (values may be rounded if desired): 

• Total Flight Hours: 

• Hours as Pilot-in-Command: 

• Hours as Second-in-Command: 

• Hours as Flight Engineer: 

• Total Years Flying: 

• Long Range Over Water, 3 or 4 engine, number of Years/Hours: / 

• Long Range Over Water, 2 engine, number of Years/Hours: / 

Military Experience 

2. Do you have any military flying experience? (Circle one.) 

Yes No If Yes, number of Years/Hours: / 

Business/Corporate Experience 

3. Do you have any business/corporate flying experience? 

Yes No If Yes, number of Years/Hours: / 

Scheduled Airline Experience 

4. How many years and hours of airline flying have you completed: / 

5. What is your current position? (Circle one.) Captain First Officer Flight Engineer 

6. Have you ever flown trans-oceanic routes? Yes No 

(If “yes,” proceed to question #6a; if “no,” skip to question 7 on p.9). 

a. On a scale of 0 to 10, rate your level of familiarity with flying oceanic routes. 

A rating of 0 corresponds to “very unfamiliar.” 

A rating of 10 corresponds with “very familiar.” 

Level of familiarity with flying oceanic routes: 
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b. What oceanic regions have you flown in? List in order of experience, with most as first. 


c. Considering the trans-oceanic route that you fly most frequently, how many times do you request 
altitude changes during a typical flight, and what percentage of the time are your altitude change 
requests granted? 

Number of altitude change requests during a typical flight: 

Percentage of time that altitude change requests are granted during a typical flight: 

d. While in Oceanic airspace do you expect ATC will allow you to climb while not in radar contact? 

Yes No 

e. What kinds of techniques do you use to facilitate your ability to get the best Oceanic crossing 
altitude? 


f. While operating beyond VHF range of ATC, have you ever coordinated Oceanic Climbs with other 
proximate aircraft on a common VHF frequency (123.45) before asking ATC for a climb via HF or 
CPDLC? Yes No 

If so, how often have you done so? 

7. What route(s) do you currently fly? 
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8. What type of equipment do you currently fly? 


9. Is your typical aircraft equipped with ADS? Yes No Sometimes Don’t Know 

10. Is your aircraft equipped with CPDLC? Yes No Sometimes Don’t Know 

1 1 . Is HF radio your primary means of communicating with ATC in Non-Radar environments? 

Yes No Sometimes 


12. Do you use ETOPS procedures on your current aircraft? Yes No 

13. How do you determine what altitude to request for an oceanic crossing? (select all that apply) 

Flight Plan 
FMC Cruise page 

If another method is used please describe: 


14. When requesting an Oceanic crossing altitude do you typically request an altitude higher or lower than 
optimum altitude displayed on the FMC cruise page? 

Higher Lower 

Experience with Vertical Situation Displays 

15. Have you ever used a vertical situation display (VSD)? Yes No 

(if “yes,” proceed to question #15a; if “no,” skip to question #16 on p.l 1) 

a. How would you characterize your previous experience(s) using VSDs? 

Very Positive Somewhat Neutral Somewhat Negative Very 

Positive Positive Negative Negative 
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Please explain: 


Experience with Electronic Flight Bags 
16. Have you ever used an EFB? 

Yes No 

(if “yes,” proceed to question #16a; if “no,” skip to question #17 on p. 12) 

a. What class(es) of EFB(s) have you used? (Please circle all that apply.) 

Class 1 EFBs [i.e., Commercial-Off-The-Shelf (COTS)-based systems, including laptop computers, 
that are fully portable] 

Class 2 EFBs (i.e., COTS-based systems that are portable, are connected to the aircraft during 
normal operations, and require an administrative control process for removal) 

Class 3 EFBs (i.e., installed equipment) 

b. How would you characterize your previous experience(s) using EFBs? 

Very Positive Somewhat Neutral Somewhat Negative Very 

Positive Positive Negative Negative 

Please explain: 
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c. Please describe the functionality of the EFB application(s) you have used most frequently in the past: 


Miscellaneous Information 


17. On a scale of 0 to 10, rate how often you use a personal computer. 

A rating of 0 corresponds to “I never use a personal computer.” 

A rating of 10 corresponds with “I use a personal computer multiple times every day.” 

Level of personal computer usage: 

18. What is your current age? 

19. Would you be willing to discuss your comments regarding the ITP Evaluator display interface with 
someone working on this research project? 

Yes No 

If "yes,” please provide your preferred contact information (i.e., name, phone, email address, etc.): 


Please note that the information collected from individual survey respondents will be kept strictly 
confidential. At no time will your individual responses (including those provided in person, by phone, or via 
email) be released to anyone other than individuals working directly on the project without your written 
consent. The information you provide will not have your name associated with it; only a participant number 
(i.e., Participant #1, Participant #2, etc.) will identify you during analyses and any written reports regarding 
the results of this survey. 

20. Would you be interested in participating in future ITP simulation experiments? 

Yes No 

If “yes,” please complete the on-line questionnaire located at: 
http://flight-research.larc.nasa.gov/ , and/or contact: 

Jennifer L. Murdoch 

Aircraft Operations & Evaluation Branch 

Mail Stop 156A, NASA Langley Research Center, Hampton, VA 23681-2199 

Phone: (757) 864-8304 

Email: Jennifer.L.Murdoch@nasa.gov 
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Appendix C 

ITP Display Interface User Survey: Overview of Results 


As described in Appendix B, the purpose of the “In-Trail Procedure (ITP) Evaluator Display User Survey” 
was to collect commercial airline pilots’ “ideas and opinions regarding the design of an ITP display 
interface.” The survey was distributed to approximately 1,500 oceanic line pilots, and 245 pilots completed 
and returned their surveys by mail to NASA Langley Research Center. Pertinent characteristics of the 
survey’s respondents and an overview of the survey’s results are provided below. 

The 245 survey respondents consisted of 97 commercial airline captains and 148 first officers who ranged 
in age between 35 - 59 years [Mean ( M) = 49; Standard Deviation ( SD ) = 7]. On average, these pilots had 
approximately 28 years (M = 28.7, SD = 6.9) and 10,500 hours of airline flying experience (M = 10,518, SD 
= 4,633). Over 75% of the pilots had experience flying in the North Atlantic and South Pacific track 
systems, and 13% had experience flying oceanic routes in the Caribbean and Polar regions as well as in 
Southeast Asia. With respect to flying long range over water, the survey respondents had, on average, 6.5 
years ( M = 6.5, SD = 7.7) and 2,800 hours (M = 2,800; SD = 3,760) of experience flying three or four 
engine aircraft and had, on average, 4.5 years (M= 4.5, SD = 3.5) and 2,390 hours (M = 2,390; SD = 2,137) 
of experience flying two engine aircraft. When asked to rate their level of familiarity with flying oceanic 
routes on a scale from 0 (i.e., “very unfamiliar”) to 10 (i.e., “very familiar”), the mean response was 9.44 
(SD = 1.22). When asked to rate their level of computer usage on a scale from 0 (i.e., “I never use a 
computer”) to 10 (i.e., “I use a computer multiple times every day”), the mean response was 8.86 (SD = 

I. 85). 

The survey respondents overwhelmingly preferred the “Combined with Vertical-on-Bottom, Track Up” ITP 
display interface, presented in Figure 3a of the survey’s introductory materials Appendix B, with 39.18% of 
the pilots providing this display format with a rank of “1” thereby indicating that it was the format that they 
liked the most. 

The “Combined with Vertical-on-Bottom, Track Up” ITP display configuration, combines a “vertical” view 
(i.e., a two dimensional view of airspace providing multiple altitudes with no graphical lateral or track 
information) and a “horizontal” view (i.e., a two dimensional view of airspace providing “look-down” 
perspective with no graphical altitude information) on a single display screen. The vertical view is 
presented below the horizontal view, and the horizontal view is presented as “track up” (i.e., an upward 
presentation of path). 

The “Combined with Vertical-on-Top, Track Up” ITP display format (Figure 4a in Appendix B), which 
combines a vertical view and a horizontal view on a single display screen and presents the vertical view 
above the “track up” horizontal view, was selected by 22.86% of the survey respondents as their favorite 
display format. The remaining four display configurations presented in the survey’s introductory materials 
(Appendix B) included: 1) the ’’Vertical-toggle-Horizontal, Track Up” display format, 2) the “Combined 
with Vertical-on-Bottom, Track Right” display format, 3) the “Combined with Vertical-on-Top, Track 
Right” display format, and 4) the ’’Vertical-toggle-Horizontal, Track Right” display format. Each of these 
display configurations was ranked as “1,” or as being the “most liked” display format, by 13.47%, 12.24%, 

I I. 43%, and 0.82% of the survey respondents, respectively. 

When asked how useful their preferred display format would be for supporting the execution of ITP flight 
level change maneuvers, 78% of the survey respondents indicated that they would find their preferred 
display format to be “very useful.” Twenty-one percent of the pilots reported that their preferred display 
format would be “useful,” and the remaining 1% reported that their preferred display format would be 
“somewhat useful” or “marginally useful” during the execution of ITP flight level change maneuvers. 
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Ninety-five percent of the survey respondents indicated that the inclusion of the horizontal view enhanced 
the ITP display, and 75.51% preferred that the horizontal view be presented as “track up” rather than “track 
right.” When asked how useful it would be to have the ability to simplify the display interface by 
requesting that either a “climb only” or a “descent only” option be depicted, 57% of the pilots indicated that 
they would find this feature to be “very useful” or “useful”; 38% indicated that they would find this feature 
to be “somewhat useful” or “marginally useful”; and 5% indicated that they would find this feature to be 
“not at all useful.” 

Survey respondents were also asked to provide feedback regarding specific aspects of the ITP display 
interface symbology. With respect to the display’s use of color, 77.5% of the survey respondents indicated 
that it was either “very appropriate” or “appropriate”; 9% indicated that it was “somewhat appropriate”; 
11% held a “neutral” opinion; 2% indicated that the display’s use of color was “somewhat inappropriate”; 
and 0.5% indicated that the display’s use of color was “inappropriate.” With respect to the information 
included in the traffic aircrafts’ data tags, 92% of the pilots indicated that it was either “very appropriate” or 
“appropriate”; 6% indicated that it was “somewhat appropriate”; 1.5% held a “neutral” opinion; and 0.5% 
indicated that the information included in the traffic aircrafts’ data tags was “inappropriate.” Ninety percent 
of the survey respondents found that having the word “unavailable” included in a traffic aircraft’s data tag, 
when applicable, was helpful while 10% did not. Lastly, with respect to the intuitiveness of the display’s 
presentation of ground speed differential, 62% of the survey respondents indicated that it was either “very 
intuitive” or “intuitive”; 23% indicated that it was “intuitive”; 5% held a “neutral” opinion; 7% indicated 
that the ground speed differential’s presentation was “somewhat unintuitive”; and 3% indicated that it was 
either “unintuitive” or “very unintuitive.” 
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Appendix D 

Flight Manual Bulletin 


INSTRUCTIONS TO SUBJECT PILOTS, NASA LANGLEY OCEANIC IN-TRAIL PROCEDURE 

EXPERIMENT, SEPTEMBER 2006 



FLIGHT MANUAL BULLETIN 


2006-01 


Insert following the BULLETINS tab. 
Revision 1.1 September 6, 2006. 


OCEANIC IN-TRAIL PROCEDURE 
Background 

Aircraft in oceanic and remote non-radar airspace frequently fly for extended periods of time in the same direction 
and along similar flight paths as other aircraft. Oceanic controllers use procedural separation to ensure 
aircraft remain separated in this airspace, typically requiring much greater separation times or distances 
between aircraft than when in radar airspace. During these long flight segments, aircraft may enhance 
operational efficiency and safety through a change of flight level by climbing (usually to increase fuel 
efficiency) or descending (usually to avoid turbulence or unfavorable winds). However, in many situations 
the standard longitudinal separation does not exist at one or more higher or lower flight levels, but does 
exist at a succeeding flight level. An aircraft desiring a flight level change to such a succeeding flight level 
would therefore be prevented or “blocked” from making the climb or descent through the intervening flight 
level(s). 

NASA is developing and testing in simulation an In-Trail Procedure (ITP), which would increase the 
opportunities for such flight level changes that would otherwise be blocked. The ITP employs new onboard 
avionics equipment that provides the crew with improved information about nearby traffic, and new 
procedures that enable the crew, when appropriate criteria are met, to request an ITP flight level change 
referencing one or two of these nearby aircraft that might otherwise block the flight level change. The ITP 
equipment uses airborne surveillance data broadcast from nearby aircraft that yields more accurate position 
data than that available to oceanic controllers, enabling controllers to approve ITP flight level change 
requests that reference these aircraft, even if standard separation would not otherwise exist with these 
reference aircraft. That is, the availability of more accurate airborne surveillance data enables safe flight 
level changes through intervening flight levels with lower separation minima than when using current 
ground-based non-radar separation rules. 

System description 

The onboard ITP equipment receives Automatic Dependent Surveillance - Broadcast (ADS-B) data from 
aircraft within reception range that are broadcasting these data. The received data includes flight id, 
altitude, position, groundspeed and quality-of-data information. The ITP equipment receives the ADS-B 
data from these aircraft and along with onboard navigation data calculates appropriate separation 
information for these aircraft. 

The ITP equipment portrays the information derived from received ADS-B data on the traffic displays of an 
Oceanic Operations application, which runs on the Electronic Flight Bag (EFB). TCAS-derived traffic data 
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are also portrayed for additional situation awareness. Both planform and profile views of traffic are 
provided, with symbols and data tags depicting the relative position, separation information, and flight id id 
of aircraft for which these data are available. Additional labels are included in the aircraft data tags, as 
appropriate, to indicate the quality of the received data or if the criteria are met to reference that aircraft in 
an ITP flight level change request. The crew controls the display of this traffic information through the 
EFB interface, and uses the information presented to decide whether, and how, to request an ITP flight level 
change. 

Operational information 

The crew uses the EFB’s Oceanic Operations application to select the desired flight level, either above or 
below the current flight level. This desired flight level can be up to 4000 feet from the current flight level. 
Once the desired flight level is selected, the application will display aircraft symbols and data tags for all 
aircraft at the current flight level, desired flight level, and intervening flight levels that are transmitting 
position information. 

Using this traffic information, the crew determines whether to make a flight level change request to the 
desired flight level, and if so, whether to request a standard or ITP flight level change. If the desired flight 
level appears available and adequate separation would appear to exist at intervening flight levels, then a 
standard flight level change could be requested. However, if potentially blocking aircraft are observed on 
the intervening flight levels, then the crew should evaluate the available information for these aircraft to 
determine if they can be used as reference aircraft in an ITP flight level change request. 

An aircraft at an intervening flight level can be used as a reference aircraft if it meets the following: 


• Same Direction of flight as the Ownship, +/- 45 degrees, 

• Qualified ADS-B Data are being received from the aircraft, and 

• ITP Distance/Speed Criteria are met. 


The ITP Distance is calculated by the Oceanic Operations application for each aircraft with this calculation 
performed as shown in the following figure (for identical ground tracks, ITP Distance is simply the distance 
between the two aircraft). The ITP speed criterion is a groundspeed differential, also calculated by the 
application and is simply the difference in groundspeed between the two aircraft. A closing groundspeed 
differential is one in which the aircraft are getting closer together. 



Figure 1 - ITP distance (non identical ground tracks vs. identical ground tracks) 
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A reference aircraft must meet these ITP Distance/Speed Criteria: 

• ITP Distance at least 15 nautical miles, and closing groundspeed differential of 20 knots or less, 
or 

• ITP Distance at least 20 nautical miles, and closing groundspeed differential of 30 knots or less. 


Up to two reference aircraft can be included in an ITP flight level change request to ATC. If more than two 
potentially blocking aircraft meet the criteria for reference aircraft, then the crew would identify the one or 
two that, in their judgment, would be most likely to block the flight level. Typically this would be the 
closest one or two such aircraft, either ahead of and/or behind the Ownship aircraft, and on either one or 
two intervening flight levels, depending on the situation. 

ATC can either: 

1) deny the ITP flight level change request due to traffic or other constraints, 

2) approve a standard flight level change if sufficient separation exists and ITP clearance is not 
necessary, or 

3) issue an ITP flight level change clearance, identifying the reference aircraft id’s 

If an ITP clearance is received, then the crew must reassess the reference aircraft identified in the 
clearance to assure that the ITP criteria are still met before accepting the clearance. If the criteria are no 
longer met then the clearance must be rejected. 

Once an ITP clearance has been accepted, the crew should commence the flight level change without delay 
and maintain cruise Mach number and at least 300 fpm vertical speed throughout the flight level change. 
If this minimum performance cannot be maintained, then regional contingency procedures for inability to 
conform to an ATC clearance should be followed. 

No further reference to the Oceanic Operations application is required after the climb or descent has been 
initiated. 


ITP data link request syntax 

ITP flight level change requests are made via data link in a manner similar to a standard flight level change 
request, but with additional ITP-specific information entered in the free text fields. This information may 
be entered on more than one free text line if necessary, but the keyword “ITP” should start the first (and 
only the first) free text line of the flight level change data link request. 

Information about each of the one or two reference aircraft is entered in the free text field after the ITP 
keyword. Information is entered regarding the relative position, flight id, and ITP Distance of each 
reference aircraft, in the following format: 


F Preference aircraft flight id>! nn or \,!<reference aircraft flight id>! nn 


Where F/ means that the Ownship aircraft is following this reference aircraft, 

L/ means that the Ownship aircraft is leading this reference aircraft, and 
/nn is the ITP Distance for this reference aircraft, in nautical miles 
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For example, a flight level change data link request with two free text lines of 


ITP F/UAL123/65 
L/DAL456/30 

would be interpreted as an ITP flight level change request with two reference aircraft: following (i.e., 
behind) UAL123, which is at an ITP Distance of 65 nm, and leading (in front of) DAL456, at an ITP 
Distance of 30 nm. 

Since there is insufficient text space to include an ITP request with two reference aircraft on a single free 
text line, the information for the second reference aircraft should be entered on the second free text line. 

Note also that if two reference aircraft are identified in an ITP request, all possible geometries are 
permissible as long as both aircraft meet the criteria for reference aircraft. That is, it is permissible to be 
following both reference aircraft, or leading both, or following one and leading the other. It is also 
permissible for the reference aircraft to be at a different intervening flight level. 
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ASTOR controls and indicators 

The Oceanic Operations experiment will be conducted using the NASA LaRC Aircraft Simulation for 
Traffic Operations Research (ASTOR) simulator. ASTOR is a medium-fidelity human-in-the-loop 
computer workstation-based aircraft simulation, which supports research of air traffic operations within 
future airspace environments. ASTOR currently simulates a Boeing 777 like interface, with aerodynamic 
performance that is more representative of a Boeing 757 transport aircraft. 


ASTOR components include (figure 2): aircraft and engine models; autopilot and autothrottle systems; 
flight management computer (FMC) and multi-function control display unit (MCDU); mode control panel 
(MCP) and electronic flight instrumentation system (EFIS) control panel; displays such as the primary 
flight display (PFD), navigation display (ND), and engine indication and crew alerting system display 
(EICAS); sensor systems; an Electronic Flight Bag (EFB) and a sophisticated simulation model of ADS-B. 



Primary Flight 
Display 


Navigation 

Display 


Radio Tuning Panel 


Transponder Control 
Panel 


Speed brakes, 
Throttles, Flaps & 
Gear 


Simulation 
Status Panel 


EFIS Control Panel 


Mode Control Panel 


Multi-function 

CDU 


EICAS & MFD Display Select Panel 


Electronic Flight 


Figure 2. Aircraft Simulation for Traffic Operations Research (ASTOR) 

The key components for the Oceanic Operations experiment are the EFB, the MFD and the Display Select 
Panel (DSP). The Display Select Panel is used primarily to cycle between the shared EICAS and MFD 
windows, using the ENG and COMM buttons respectively, while the MFD windows are used for data link 
communications with ATC. 
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Electronic Flight Bag 

The Electronic Flight Bag (EFB) is a multi-function display, which can enable many different applications. 
For the purpose of this experiment, only the functionality of the Oceanic Operations application (figure 3) 
will be described. 

The Oceanic Operations application was designed to present surveillance information about the surrounding 
traffic to the flight crew. The information, which is presented on both a horizontal planform and vertical 
profile view, includes the relative positions of other aircraft and other traffic information (e.g., call-sign, 
relative altitude, etc.) needed to help the crew to maintain a high level of traffic awareness and to allow 
them to evaluate a possible flight level change. 

Simulated line select keys on the edge of the display can be used for selecting certain functions and/or 
display modes. Some of these functions and modes can also be selected by touching the adjacent screen 
icon using the stylus/mouse. 


Planform 

View 



Line Select key 


Screen icon 



\ 

Screen icon 


Vertical 
Profile View 


Figure 3. EFB OCEANIC OPERATIONS application 


Note: The display may NOT be used for navigating the aircraft. It does NOT have position data, heading, 
air speed or ground speed information. 
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Menu pages 

The default page of the EFB is the MAIN MENU page (figure 4). It is displayed whenever the EFB is 
turned on. This page provides selections to multiple sub-pages of which only the OCEANIC 
OPERATIONS page has been implemented. 

To access the Oceanic Operations application, select the APPLICATIONS MENU and subsequently 
OCEANIC OPERATIONS (figure 5). 
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Figure 4. EFB main menu 


fBRT 

hr 

MENU 

M) 

PGUP 

PGON XFR 

ENTER 

PWR 

•w 

[DIM 


Afj 

’PLICATIONS MENU 










EE)- 

ARRIVAL 

SPAC 1 NG 




-E 








El- 

DYNAMIC 

A 1 RWAY 




‘B 

E ( 

OCEANIC 

OPERATIO^^ 



-El 

Eh 






-B 

Eh 






"• [ ~ J 

Eh 






-B 








Eh 






-r- 







fr 

Eh 






-B 


E 

lffoT| 

tTJ 

[T) B 

0 



Figure 5. EFB applications menu 


The MENU button or subsequent selections 
of the back arrow (<-) bezel button can be 
used to return to the MAIN MENU. 
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OCEANIC OPERATIONS application 



Figure 6 - Oceanic default display 
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Figure 7. Oceanic descent display with all tracks 


The default display (figure 6) presents surveillance data of all 
(ADS-B and TCAS) aircraft that are on the same track as the 
Ownship. 

Ownship is depicted by a white triangle on both the planform and 
profile view of the display. 

On the planform view a dashed rose is available that can be used 
for relative angular position information. A white line, stretching 
out in front and behind the Ownship includes tick marks, to 
indicate range in Nautical Miles. 

On the profile view, horizontal dashed lines show subsequent 
Flight Levels. Tick marks again indicate range to Ownship in 
Nautical Miles. 

The MCP selected flight level is indicated by a solid magenta line. 
A dashed green line indicates the selected desired flight level. 

To increase the value for the desired flight level, press the 
INCR FL bezel button or use the stylus/mouse to touch the screen 
icon. To decrease the value for the desired flight level use the 
DECR FL. Note that the selection cannot be increased or 
decreased beyond the ITP limit of 4,000 feet nor can it be within 
less than 1000ft of the current flight level. A cyan box around the 
screen icon will indicate the user has reached this limit. When 
changing the desired flight level, additional ITP information will 
appear on the data tags for those aircraft that are between the 
current and desired flight level. 

The range of both the planform and profile view can be adjusted 
by pressing the ZOOM IN or ZOOM OUT bezel button or by 
using the stylus/mouse to touch the screen icon at the bottom of 
the display. The tick marks will indicate the new scale. A 
box around the screen icon will indicate the zoom limit. 

By default the display will come up in Climb View mode, 
indicated by the Ownship residing on the lowest flight level of the 
profile view. This mode enables flight level changes to a higher 
altitude. When a lower flight level is desired, the profile view can 
be changed to a Descent View mode (figure 7) by pressing the 
VIEW DES bezel button or using the stylus/mouse to touch the 
screen icon. In this mode the Ownship will reside on the highest 
flight level. To return to the climb mode select VIEW CLB. 

For enhanced situation awareness, select the VIEW ALL bezel 
button or use the stylus/mouse to touch the screen icon. This view 
will enable additional aircraft to be drawn that are not on the 
Ownship track. The additional aircraft will only be displayed on 
the planform view. To return to the track only view select VIEW 
TRACK. 
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Planform View Symbol 


Meaning 


Ownship 


The aircraft you are flying. 


ADS-B aircraft 


Potentially blocking 
ADS-B aircraft 


TCAS-only aircraft 


(Blocking) 
TCAS-only aircraft 


A cyan chevron with the tip of the chevron indicating the 
direction of travel. The altitude relative to Ownship is 
indicated by a numerical value (in hundreds of feet) right 
above or below the chevron. Example aircraft is 1,000 
feet above Ownship. The absolute altitude will be 
displayed when ABS is selected on the Transponder 

Control Panel. 

A cyan chevron with the tip of the chevron indicating the 
direction of travel. The altitude relative to Ownship is 
indicated by a numerical value (in hundreds of feet) right 
above or below the chevron. The aircraft call-sign / flight 
identifier will be displayed for all ADS-B aircraft that 
reside from the MCP selected up to and including the 
desired altitude. Additional information for this aircraft 

is presented in the data tag on the profile view. 

A cyan diamond indicates that it is a TCAS-only aircraft. 
The altitude relative to Ownship is indicated by a 
numerical value (in hundreds of feet) right above or 
below the diamond. Example aircraft is 2,000 feet below 
Ownship. The absolute altitude will be displayed when 
ABS is selected on the Transponder Control Panel. 


A cyan diamond indicates that it is a TCAS-only aircraft. 
NO ADS-B (TCAS-only) aircraft are considered to be 
blocking aircraft when they reside from the MCP 
selected up to and including the desired altitude. 


Example 



+ 10 
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Profile View Symbol 

Meaning 

Example 

Ownship 

The aircraft you are flying. 

s 

ADS-B aircraft 

A cyan chevron with the tip of the chevron indicating the 
direction of travel. Aircraft equipped with ADS-B will show a 
call-sign / flight identifier (CDN814). 




CDN8 14 


Potentially blocking 
ADS-B aircraft 
In front & Separating 

Potentially blocking aircraft are all qualified ADS-B aircraft 
that reside from the MCP selected up to and including the 
desired altitude. The open triangle indicates that this aircraft 
is moving away from Ownship. In this case, VIR884 is 21 
Nautical Miles away and pulling away from Ownship at 
9KTs. 


B8B1 
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Potentially blocking 
ADS-B aircraft 
Behind & Closing 


Potentially blocking aircraft are all qualified ADS-B aircraft 
that reside from the MCP selected up to and including the 
desired altitude. The closed triangle indicates that this aircraft 
is closing in on Ownship. In this case, SAS316 is 26 Nautical 
Miles away and is closing on Ownship at 8KTs. 


--jv-..- 

SAS316 
26NM ► 8KT 


Potentially blocking 
ADS-B aircraft 
Behind & Separating 


Potentially blocking aircraft are all qualified ADS-B aircraft 
that reside from the MCP selected up to and including the 
desired altitude. The open triangle indicates that this aircraft 
is moving away from Ownship at 2KTS. 



Potentially blocking 
ADS-B aircraft 
In front & Closing 


Potentially blocking aircraft are all qualified ADS-B aircraft 
that reside from the MCP selected up to and including the 
desired altitude. The closed triangle indicates that this aircraft 
is closing in on Ownship at 9KTs. 



ADS-B aircraft 
NO REF 


Potentially blocking aircraft that do not meet the ITP criteria. 
These NO REF aircraft cannot be used as reference aircraft 
for an ITP request. In this case the closure rate is greater than 
30KTs. 


COA337 

19NM ► 39KT 


NO REF 


TCAS-only aircraft 


TCAS-only aircraft do not broadcast a call-sign / flight 
identifier, this is indicated by NO ADSB in the data tag. 


TCAS-only (blocking) 
aircraft 


TCAS-only (NO ADS-B) aircraft do not meet the ITP criteria 
for reference aircraft and are considered to be blocking 
(indicated with the NO REF data tag) when the aircraft reside 
from the MCP selected up to the desired altitude. These 
aircraft cannot be used for the ITP. 


Opposite direction aircraft 


Aircraft that is flying on the same track as Ownship but from 
an opposite direction. 



Commanded Altitude 


Altitude selected on the MCP. 



Desired Altitude 


Desired altitude selected on the EFB. 


350 — - v 

60 
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Multi Function Display 

The Multi Function Display (MFD) for the OCEANIC OPERATIONS experiment is primarily used for 
Data-Link communication with the Air Traffic Service Provider. Pressing the COMM button on the 
Display Select Panel (figure 8) will give access to the communications menu on the MFD (figure 10). To 
return to the primary engine instruments display, press the ENG button (figure 9). 



Figure 8. Display Select Panel 
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Figure 9. Primary Engine display Figure 10. MFD comm, menu 

Using the mouse and clicking on the appropriate MFD button will give access to the available sub-pages. 
Only pages that are required for the OCEANIC OPERATIONS experiment have been fully implemented. 
Other pages will either be unavailable (indicated by a cyan color) or empty (figure 11). Certain pages also 
include an upper and lower part, which can be viewed by clicking the bar, which will become available on 
the right-hand side. 


An uplink message from ATC will be displayed in the lower left-hand side of the primary engine 
instruments display (figure 12). The reception of a message will be accompanied by an aural sound. 
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Figure 11. ATC menu 



Figure 12. ATC uplink message 


Note: To conserve display space, the ASTOR has been configured to switch the normal Upper EICAS 
display on the 777 with the MFD. This causes the primary engine instruments to disappear whenever the 
COMM menu is selected. 
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Example: ITP climb request 

The subsequent chapter will describe an example of the OCEANIC IN-TRAIL PROCEDURE in the 
ASTOR. This example will include all of the steps required to complete the ITP procedure. 


Step 1: Determine desired altitude 

The first step in the procedure is to determine whether a flight level change is desired. On an actual aircraft 
there might be many reasons that determine whether to request a flight level change. In this experiment the 
scenarios are designed such that the recommended flight level on the VNAV page of the Multi-function 
Control and Display Unit (MCDU) indicates which flight level is 
currently desired. To obtain the recommended flight level press 
VNAV on the MCDU. 

The operation of the MCDU on the ASTOR is very similar to 
MCDU’s found on Boeing aircraft. Figure 13 depicts the VNAV 
Cruise page for an aircraft enroute to Frankfurt Germany that is 
currently cruising at FL330. As indicated by the optimum flight level 
indicator, this aircraft is currently 2,200 feet below its optimum flight 
level while FL350 is recommended (figure 13). 

Step 2: Select desired Flight Level on EFB 

After determining that a flight level change is desired, select the 
desired flight level on the EFB. Press the INCR FL button until the 
green dashed line lines up with the desired flight level (in this case 
FL350). Aircraft that reside on the desired or a flight level between the 
current flight level and the desired flight level will now show 
additional data in their data tags (figure 14). This data can be used for 
the ITP request. 


Step 3 Identify Potentially Blocking Aircraft 

On the EFB, look for aircraft that might 

potentially block a climb request. If the desired 

flight level is blocked, do not make the request. 

In this example, two aircraft (LTU401 and 

MAH714) potentially block the climb request. 

No aircraft is blocking the desired flight level. 

t-i • 1A i • A u .. A ! .. DAL8 1 8 is beyond the desired FL, so no speed or 

Figure 14 - EFB desired altitude selection J 5 r 

distance information is included on the data tag. 

Based on the reduced separation of the two aircraft potentially blocking the climb, a Standard Altitude 

request is not available and an ITP is required. Note: when in doubt consider an ITP request. 

Step 4: Evaluate for ITP criteria 

Next select from the list of potentially blocking aircraft the one or two, ahead of and/or behind the Ownship 
aircraft, and on either one or two intervening flight levels, that would be most likely to block the flight level 
change. In the example this would be LTU401 and MAH714. Observe the data tag and determine that 
neither aircraft displays a “NO REF” or “NO ADSB” tag meaning that the aircraft meet the ITP ADS-B 
qualification, distance and speed criteria. 


1 

1 

1 1 

1 

1 

1 



DAL8 18 1 

1 




LTU401 
70m ► 18KT 


n MAH714 1 
34NMI E> 8KT 


ZOOM 

ZOOM 

INCR 

OECR 

IN 

OUT 

FL 

FL 



Figure 13 - MCDU VNAV cruise page 
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Step 5: Prepare and send ITP altitude change request 

Now that you have determined that you would like to make an ITP 
request and have selected the reference aircraft, you need to construct 
and send the ITP request message. To do so, switch to the MFD 
display by selecting the COMM button on the Display Select Panel 
(figure 15). This will take you to the COMM page where you select 
the ATC icon to access the ATC communications page. 




Figure 15. Display Select Panel 


From the ATC communications page select the 
ALTITUDE REQUEST icon (figure 17), which will take 
you to the altitude request page (figure 18). This page 
consists of two separate parts, which can be accessed by 
clicking the scroll bar on the right-hand side. On the upper 
part (1) enter the desired altitude. On the lower part (2) 
select a reason for the flight level change request and enter 
in the Free Text field the data required for ITP. 



Figure 16. COMM page 


Note: It is not required to provide a reason for the flight 
level change request. 


ATC 

FLIGHT 

INFORMATION 

COMPANY 

REVIEW 

MANAGER 

NEW MESSAGES 

ATC 

ALTITUDE 

REQUEST 

WHEN CAN WE 
EXPECT 

EMERGENCY 

REPORT 

ROUTE 

REQUEST 

VOICE CONTACT 
REQUEST 

ATC REQUESTED 
REPORTS. . . 

SPEED 

REQUEST 

LOGON/STATUS 

POSITION 

REPORT 

CLEARANCE 

REQUEST 


FREE TEXT 
MESSAGE 


EXIT 

MENU 


ATC 

REVIEW 


FLIGHT 

INFORMATION 


0 1 3 1 z 


MANAGER 

ALTITUDE REQUEST 


ALTITUDE: 

<0 STEP AT: 


<0 CRUISE CLIMB 


BLOCK: 


TO: 


REQUEST VMC DESCENT 


COMPANY 


NEW MESSAGES 


PRINT RESET RETURN EXIT 


Figure 17. ATC COMM page 


Figure 18. Altitude Request page 
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Fisure 19. MCDU 350 in scratchnad 


Figure 20. MFD FT ,350 in altitude window 


To enter the desired flight level, click on the MCDU and enter a value in the MCDU scratchpad (figure 19). 
As the ASTOR is a window-based simulation, the MCDU must be “clicked” to make the MCDU the active 
window. To transfer the data from the MCDU scratchpad to the ALTITUDE window click on the blank 
altitude entry box of the MFD (figure 20). 

As on the 777 the reason for climbing can be checked (optional). The Oceanic ITP was designed to 
facilitate altitude changes DUE TO AIRCRAFT PERFORMANCE or DUE TO WEATHER. AT PILOT’S 
DISCRETION and MAINTAIN OWN SEPERATION AND VMC are NOT valid options for requesting an 
Oceanic ITP. 


As described in the ITP data link request syntax section of this FMB, an ITP is requested by specially 
formatted FREE TEXT. Enter the text on the MCDU scratchpad (figure 21) similar to what was done with 
the altitude and click on the FREE TEXT line to transfer this information (figure 22). For the first aircraft, 
you enter: ITP L/LTU401/70 
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PRINT RESET RETURN EXIT 


Figure 21. MCDU reference aircraft 1 
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Figure 22. MFD reference aircraft 1 
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Ficnire 21 . MCDTT reference 


Figure 24. MFD reference aircraft 2 


In an ITP with two reference aircraft, the information for the second reference aircraft needs to be entered 
on a separate line. The text is again entered on the MCDU scratchpad (figure 23) and transferred to the 
MFD by clicking on the second FREE TEXT line (figure 24). Note that the second line of FREE TEXT is 
NOT preceded by TTP\ For the second aircraft enter: F/MAH714/34 

When the ALTITUDE REQUEST FORM is completed click the SEND button. 

After the ITP request is sent, expect a STANDBY message from ATC (figure 25). 

ATC will send a reply after ATC has evaluated the request. In the example, you received a clearance for an 
ITP CLIMB TO FL350 LEADING LTU401 AND FOLLOWING MAH714 (figure 26). The message also 
includes the instruction to report reaching FL350. 
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Figure 25. MFD STANDBY 


Figure 26. MFD ITP Clearance 
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Figure 27. MFD Messages list 


Figure 28. MFD Message information 


Step 6: Retrieve message 

To retrieve the information from the ATC message, click the COMM button on the Display Select Panel. 
Clicking this button will take you to the NEW MESSAGES page. Here you will find a list of all available 
messages (figure 27). If only one message is available, you will immediately be directed to the additional 
information of that message (figure 28), otherwise click on the message to retrieve the additional 
information. 

ACCEPT and REJECT buttons appear in the lower left and right corner of the ATC message page. At this 
point ACCEPT or REJECT the instruction. Before doing this, first reassess the ITP instruction. 

Step 7: Reassess ITP Criteria 
The crew must reassess the reference aircraft 
identified in the clearance to assure that the ITP 
criteria are still met before accepting the clearance. 

The clearance must be rejected if the criteria are no 
longer met. 


In the example the aircraft referenced by ATC in the 
clearance are LTU401 and MAH714. When you 
examine their data tags on the EFB, you see that 
both aircraft are still meeting the ITP range and 
speed criteria (figure 29). The clearance has to be 
rejected if any of the aircraft, referenced by ATC, do 
not meet the criteria. A “NO REF” or “NO ADSB” 
data tag will indicate this. 

The next step is to accept the ITP clearance. 


DAL818 


MAH7 1 4 n 
34NM l>8KT 




LTU401 
70m ► 18 KT 


Figure 29. EFB Reassess 
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Figure 31. MFD ATC MESSAGE ACCEPTED 


Figure 30 - MFD ATC MESSAGE ACCEPT 

Step 8: Accept (or reject) clearance 

To accept the IPT clearance, click the ACCEPT icon in the lower left corner of the display (figure 30). To 
reject click the REJECT button in the lower right comer. When the accept message is sent, the word 
ACCEPTED will be displayed in the upper right corner and a DISPLAY REPORT button will appear 
replacing the ACCEPT button (figure 31). When pressed, this button takes us to the ATC Report page, 
where you arm the ATC report. 


In order to cancel a request, click the FREE TEXT MESSAGE icon in the lower right comer of the main 
ATC menu page (figure 11). To cancel the request, the free text has to include the word ‘CANCEL’ or 
‘DISREGARD’. Free text is entered into the MCDU scratchpad (figure 32) and transferred to the Free Text 
line by clicking on the line (figure 33). After the text has been entered click the send button and wait for a 
reply from ATC. 
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Figure 32. MCDU Cancel 
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Figure 33. MFD Cancel 


Note: Keep in mind that requests can only be canceled before an instmction has been received. When an 
instmction is received you need to follow that instmction. 
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Step 9: Arm report message 

For the purpose of closing the maneuver, you need to send a report to ATC when established on the new 
altitude. To arm this report, click ARM on the ATC REPORT page (figure 34). The word ARMED will 
appear in the upper right corner (figure 35). Upon meeting the report condition an automatic report will be 
sent. In the example a message will be sent when the aircraft is established at FL350. 



Figure 34. DISPLAY REPORT ARM Figure 35.- DISPLAY REPORT ARMED 
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Step 10: Commence flight level change 

Once an ITP clearance has been accepted, the crew should commence the flight level change without delay, 
while maintaining the current cruise Mach number and at least 300 fpm vertical speed throughout the 
flight level change. If this minimum performance cannot be maintained, then regional contingency 
procedures for inability to conform to an ATC clearance should be followed. No further reference to the 
Oceanic Operations application is required after the climb or descent has been initiated. 

Any method for climbing the aircraft with a minimum of 300 fpm while maintaining the current Mach 
number can be used. 

The MCP Altitude is adjusted on the ASTOR by clicking on the edge of the MCP altitude knob (figure 36). 
To increase the value in the MCP altitude window click on the right side on the altitude knob. To reduce the 
value in the MCP altitude window click on the left side of the altitude knob. The cleared flight level in the 
ATC message will turn green (figure 31) when it is identical to the MCP selected value. To enter the MCP 
altitude into the FMC click on the center of the MCP altitude knob. 


To execute a FLCH altitude change, click the FLCH button after the new altitude has been entered in the 
MCP altitude window. Maintain the required Mach number and vertical speed. 


A Vertical Speed climb is executed by clicking on the lower portion of the VS knob after the new altitude 
has been entered in the MCP altitude window. A Vertical Speed descent is executed by clicking on the 
upper portion of the VS Knob. The operational limitations of using Vertical Speed need to be observed. 



Step 11: Review messages 

Messages, which have been sent or received via Data-Link, 
can be reviewed by selecting REVIEW on the 
communications page of the MFD. To review the received 
messages click REVIEW ATC UPLINK (figure 37), to 
review the downlink messages, click REVIEW ATC 
DOWNLINK. 

The procedure is closed once established at the new flight 
level and the automated report message has been sent. 


Figure 37. REVIEW ATC UPLINK 
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OCEANIC OPERATIONS ITP CHECKLIST 


PNF Desired flight level SELECT on EFB 

PNF Intermediate flight levels . . . IDENTIFY POTENTIALLY BLOCKING A/C 
PNF Reference A/C (select a max of 2) EVALUATE FOR ITP CRITERIA 

• Same Direction 

• Qualified ADS-B Data 

• ITP Distance/Speed Criteria: 


ITP Distance at least 

Closing Ground Speed Differential 
no more than 

15 NM 

20KTS 

20 NM 

30KTS 


PNF ITP flight level change data link request PREPARE AND SEND 

Format free text field: 

ITP T/<Reference A/C flight id>/<distance> 

Note 1: Use “L”/ if you are Leading this Reference A/C, “F”/ if following 
Note 2: Enter 2 nd Reference A/C, if any, on 2 nd free text line, without ITP prefix 
Example: ITP F/TWA1 23/35 

L/TWA456/26 

On receipt of ITP clearance: 

PNF Reference A/C in ITP clearance REASSESS ITP CRITERIA 

If ITP Criteria not met then reject clearance. Otherwise: 

PF Acknowledge and comply with ATC clearance 
Monitor 

• Constant Mach 

• Minimum 300 FPM vertical speed 
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Appendix E 

Post-Scenario Questionnaire 

Please consider the flight scenario that you just completed and respond to the questionnaire items that follow. (If 
you made multiple ITP requests during the scenario that you just completed, please answer the questionnaire in 
terms of the first request that you made.) 

1 . What type of flight level change did you request? (Check one.) 

Standard Climb 

Standard Descent 

ITP Leading Climb 

ITP Leading Descent 

ITP Following Climb 

ITP Following Descent 

ITP Combined Climb 

ITP Combined Descent 

None (Please explain why you chose not to request a flight level change:) 

I did not request a flight level change because: 
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2. Was your flight level change request approved or denied by ATC? 

Approved (If your ITP flight level change request was approved, skip to Question #4) 

Denied 

For ITP Flight Level Changes Only 

3. Was it obvious from the ITP display why your flight level change request was denied by ATC? 

Yes 

No 

4. Did the procedure outlined for use during this ITP flight level change contain any incorrect, missing, 
and/or extra steps? 

Yes 

No (If No, skip to Question #6) 

5. Please complete the appropriate statement(s) below and explain your rationale if you have not already 
done so in another post-scenario questionnaire: 

a. The following procedural step(s) were incorrect: 
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b. The following procedural step(s) were missing: 


c. The following procedural step(s) were extraneous/overly specified: 


6. Did the procedural steps that you were instructed to use during this ITP flight level change occur in a 
logical sequence? 

Yes 

No (If No, please explain which steps were ordered incorrectly if you have not already done so in 

another post-scenario questionnaire:) 

I was instructed to perform the following procedural steps in an illogical sequence: 
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7. Would you want to perform this flight level change differently? 


Yes 

No 

Please explain: 


8. Would you recommend changes to the display interface to assist you with this ITP geometry? 

Yes (If Yes, please describe your recommended display changes unless you have already done so in 

another post-scenario questionnaire:) 

No 

Please explain: 
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Appendix F 

Post-Experiment Questionnaire 

Please respond to each questionnaire item that follows: 

1. Please share your impressions of the flight scenarios (e.g., comment on their level of realism, 
appropriateness, and/or diversity): 


2. Please share your impressions of the simulator (e.g., comment on the level of realism and 
appropriateness for this experiment): 


3. Did you receive adequate training with respect to: a) performing the ITP, b) flying the simulator, and c) 
using the EFB’s Oceanic application? 

Yes 

No 

Please explain and provide any suggested improvements for the training protocol: 
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4. Please share any suggested improvements for other aspects of the experiment (e.g., schedule, facilities, 
etc.): 


5. Given your operational experience, please describe any performance concerns that you have with the 
ITP (e.g., the ability to maintain climb rate or Mach): 


6. Given your operational experience, do you think this procedure would be acceptable? 

Yes 

No 

Please explain: 
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7. Did you use the Oceanic Operations ITP Checklist during every ITP maneuver? 

Yes 

No 

If “no,” please estimate the percentage of time that you used the ITP Checklist: 


8. Did you reassess the ITP criteria every time you received an ITP clearance? 

Yes 

No 

If “no,” please estimate the percentage of time that you reassessed the ITP criteria when you received 
an ITP clearance: 


9. Please describe how the workload required to operate the simulator compares with the workload required 
to fly an actual aircraft in the domain that was simulated: 
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10. Please describe how the workload required to perform standard flight level changes during the 
experiment compared with the workload required to perform ITP flight level changes: 


1 1 . Please comment on the perceived benefits that the ITP may have for a) the flight crew, b) passengers, 
and c) the airline company: 

a) Benefits that the ITP may have for the flight crew include: 


b) Benefits that the ITP may have for passengers include: 


c) Benefits that the ITP may have for the airline company include: 
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12. Compared with current day procedures, the ITP is: 

Less safe than current day procedures 

Equally as safe as current day procedures 

Safer than current day procedures 

Please explain: 


13. What did you like about the ITP display interface? 


14. What did you dislike about the ITP display interface? 


15. How would you improve the ITP display interface and why? 
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16. What is the minimum required information that should be presented on the ITP display? 


17. Was the ITP free text phraseology easy to understand and use? 

Yes 

No 

Please explain: 


18. If you encountered off-nominal conditions while performing an ITP flight level change (e.g., you 
observe the reference aircraft deviating), how easy would it be for you to revert back to the use of standard 
procedures? 

Very Easy Somewhat Undecided Somewhat Difficult Very 

Easy Easy Difficult Difficult 

Please explain your answer: 
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Appendix G 
Experiment Scenarios 
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Appendix H 

Supplemental Data Collection Scenarios 


(#17) Combined Climb through 3 Intervening (CC3_LF_AC) 


(#18) Combined Climb through 2 Intervening (CC2_LF_AC) 


Ownship: FL 330; 0.81 M 

Desired Alt: FL370 (FMS recommended 360) 


Ownship: FL 340; 0.82M 
Desired Alt: FL370 




(#19) Combined Climb through 2 Intervening (CC2_FF_AC) 


(#20) Combined Climb through 2 Intervening (CC2_LL_AC) 


Ownship: FL 340; 0.80M 
Desired Alt: FL370 


Ownship: FL 340; 0.81 M 
Desired Alt: FL370 



4/5/2007 


EOO HITL scenario graphics v0.8 
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(#21) Combined Climb through 3 Intervening (FC3_UAT) 
(FL350 is available) 

Ownship: FL 330; 0.81 M 
Desired Alt: FL370 


(#22) Combined Climb through 3 Intervening (CC3_LF_UAD) 
(FL370 is available after x minutes) 

Ownship: FL 330; 0.82M 
Desired Alt: FL370 




4/5/2007 


EOO HITL scenario graphics v0.8 
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4/5/2007 


EOO HITL scenario graphics v0.8 
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Appendix I 

Informed Consent Document 


Project Title: Enhanced Oceanic Operations (EOO) Human-In-The-Loop 

In-Trail Procedure (ITP) Validation Simulation Study 

Principal Investigator: Jennifer L. Murdoch, Ph.D., (757) 864-8304, 

< J ennifer . L . Murdoch@nasa . go v> 


PURPOSE 


The purpose of NASA Langley Research Center’s (LaRC) Enhanced Oceanic Operations (EOO) 
Human-In-The-Loop In-Trail Procedure (ITP) Validation Simulation Study is to validate the 
procedures associated with normal execution of oceanic ITP maneuvers and to assess pilot 
acceptability of ITP maneuvers. 

You are invited to participate in this research because you are a Boeing 777 or 747-400 captain or first 
officer with current oceanic experience. 

Eighteen to 24 participants are expected to take part in this study. Your participation in this research 
endeavor will assist with the design and development of more efficient oceanic operations. 

PROCEDURES 

If you agree to participate, you can expect the following to occur. You and up to five other pilots (who 
meet the same criteria that you do) will be asked to participate in a training session designed to help you 
understand the tasks you will be asked to perform during the experiment. You will be given ample training 
time to become comfortable with all of the experimental tasks. 

The experiment will involve the following: 

• You will be asked to fly a series of scenarios using a desktop aircraft simulator. 

• Each scenario will be flown across the North Atlantic Ocean and will potentially involve either a 
standard or ITP flight level change request. 

• During scenarios involving ITP maneuvers, the percentage of time that you maintain cmise Mach, 
the percentage of time that you maintain a minimum climb or descent rate, and the point of closest 
approach between your aircraft and simulated traffic aircraft will be recorded. 

• After each scenario, you will be asked to complete a workload rating scale and a post-scenario 
questionnaire. 

• General comments regarding the ITP will be collected at the end of the study. 

Participation in this study will require approximately 1 8 hours of your time over the course of two 
consecutive days. A lunch period and ample breaks will be provided each day. 
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RISKS 


There are no apparent risks associated with participation in this study other than those experienced during 
normal participation in flight simulation activity. There is a very slight risk that you could confuse the 
procedures for this activity with standard Approved ICAO procedures. 

In the unlikely event that you are injured or otherwise experience discomfort, NASA LaRC has an on-site 
Occupational Health Clinic. The Clinic has hours of operation from 7:00 a.m. to 3:30 p.m. The clinic 
number is 864-3193. Emergency medical personnel and ambulance service is also available to transport 
you to nearby health care providers. 

If you have questions about the research and your rights should you experience any injury, you may contact 
the Principal Investigator listed at the beginning of this document. 


BENEFITS 

There are no personal benefits associated with participating in this study. However, your participation in 
this research endeavor will provide information that may be helpful in the design and development of more 
efficient oceanic operations. 


COSTS AND COMPENSATION 

There are no costs associated with participating in this study. You will be compensated by receiving a 
stipend of $200.00 a day, plus travel expenses. 

Non-civil servant volunteers injured as a result of participating in this research may file a claim under the 
Federal Tort Claims Act (FAC A) by filing a Standard Form 95. For additional information, you may 
contact the NASA FaRC Office of Chief Counsel at 864-3221. 


CONFIDENTIALITY 

Records of participation in this study will be kept confidential to the extent permitted by law. To ensure 
confidentiality, an arbitrary identification number will be assigned to your responses. In the event of any 
report or publication from these studies, the identity of participants will not be disclosed. Results will be 
reported in a summarized manner in such a way that you cannot be identified. 


VOLUNTARY PARTICIPATION 


Taking part in this study is voluntary. You may withdraw from participating at any time, a decision which 
will not result in any penalty or loss of benefits to which you may otherwise be entitled. 


SAFETY 

As a voluntary test subject participating in this research, I understand that: 

• NASA is committed to ensuring my safety, health, and welfare plus the safety and health of all 
others involved with this research. 
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• I should report any accident, injury, illness, changes in my health condition, hazards, safety 
concerns, or health concerns to Jennifer Murdoch at (757) 864-8304. If I am unable to reach the 
above named individual or am not satisfied with her response, I should contact the NASA LaRC 
Safety Office at (757) 864-7233 or the Chairperson of the NASA LaRC Institutional Review Board, 
Mr. Jeffrey S. Hill, at (757) 864-5107. 

• If I detect any unsafe condition that presents an imminent danger to myself, or others, I have the 
right and authority to stop the activity or test. In such cases, the Principal Investigator and 
associated research personnel will comply with my direction, stop the activity, and take action to 
address the imminent danger. 


QUESTIONS 

Questions are encouraged. If there are any questions about this study, please contact Jennifer Murdoch of 
NASA LaRC at (757) 864-8304. 

I certify that I have read and fully understand the explanation of procedures, benefits, and risks associated 
with the research described herein, and I agree to participate in the research described herein. My 
participation is given voluntarily and without coercion or undue influence. I understand that I may 
discontinue participation at any time. I have been provided with a copy of this consent statement. If I have 
any questions or modifications to this consent statement, they are written below. 

Participant’s Name (printed): 


(Signature of Participant) 


(Date) 


(Participant’s Date of Birth) 


INVESTIGATOR STATEMENT 

I have discussed the above points with the participant, using a translator when necessary. It is my opinion 
that the participant understands the risks, benefits, and procedures involved with participation in this 
research study. 

Investigator’s Name (printed): 


(Signature of Investigator) 


(Date) 
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Appendix J 

Demographic Information Questionaire 


Personal Information 

1. On a scale of 0 to 10, rate how often you use a personal computer. 

A rating of 0 corresponds to “I never use a personal computer.” 

A rating of 10 corresponds with “I use a personal computer multiple times every day.” 
Level of personal computer usage: 

2. What is your current age? 

General Experience 

3. Provide your best estimate for each of the following (values may be rounded if desired): 

• Total Flight Hours: 

• Total Years Flying Oceanic Operations: 

• Over Water, 3 or 4 engine, number of Years/Hours: / 

• Over Water, 2 engine, number of Years/Hours: / 

Military Experience 

4. Do you have any military flying experience? 

Yes No If Yes, number of Years/Hours: / 

Business/Corporate Experience 

5. Do you have any business/corporate flying experience? 

Yes No If Yes, number of Years/Hours: / 

Scheduled Airline Experience 

6. How many years and hours of airline flying have you completed: / 

7. What is your current position? Captain First Officer 

8. On a scale of 0 to 10, rate your level of familiarity with flying oceanic routes. 

A rating of 0 corresponds to “very unfamiliar.” 

A rating of 10 corresponds with “very familiar.” 

Level of familiarity with flying oceanic routes: 
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9. Approximately how many oceanic flights have you completed during the last year? 

Approximate number of oceanic flights completed during the last year: 

10. What oceanic regions have you flown in? List in order of experience, with most as first. 


1 1 . Considering the trans-oceanic route that you fly most frequently where radar service is provided, how 
many times do you request flight level changes during a typical flight, and how many of your flight level 
change requests are approved? 

Number of flight level changes requested during a typical flight: 

Number of flight level change requests approved during a typical flight: 

12. Under what circumstances would you expect to be more or less likely to obtain approval for a flight 
level change in oceanic, non-radar airspace? Consider the time of year, time of day, direction of flight, 
particular tracks, and/or particular flight levels. 


13. What kinds of techniques do you use to facilitate your ability to get the best oceanic crossing flight level 
upon entering the track system and later in the flight? 
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14. Use a scale of 0 to 10 to indicate how important it is to obtain FMS optimal flight level: a) at track 
entry, and b) at any point during an oceanic flight. Also, describe what drives this decision (e.g., airline 
policy) and indicate how often fuel efficiency is deemed to be more important than time during oceanic 
crossings. 

A rating of 0 corresponds to “very unimportant.” 

A rating of 10 corresponds with “very important.” 

a) Importance of obtaining FMS optimal flight level at track entry : 

b) Importance of obtaining FMS optimal flight level at any point during an oceanic flight: 

c) Decisions regarding optimal flight level are determined by: 


d) How often is fuel efficiency deemed to be more important than time during oceanic flights? 


15. While operating beyond VHF range of ATC, have you ever coordinated oceanic climbs with other 
proximate aircraft on a common VHF frequency (123.45) before asking ATC for a climb via HF or data 
link? 


Yes No If Yes, how often have you done so? 

16. Do you have experience with data link communications? 

Yes No If Yes, how many years of experience do you have? 

17. Do you have experience with ADS-C (i.e., data link position reporting)? 

Yes No If Yes, how many years of experience do you have? 
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18. What route(s) do you currently fly? 


19. What type of aircraft and model do you currently fly on your oceanic routes? 


20. Is your oceanic aircraft equipped with ADS-B? 

Yes No Sometimes Don’t Know 

21. Is your oceanic aircraft equipped with data link? 

Yes No Sometimes Don’t Know 

22. Is HF radio your primary means of communicating with ATC in non-radar environments? 

Yes No Sometimes 

23. How do you determine what flight level to request for an oceanic crossing? (Select all that apply.) 

Flight Plan Information Obtained from Dispatch 
FMC Cruise Page 

If another method is used please describe: 
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24. When requesting an oceanic crossing flight level, do you typically request a flight level that is the same 
as, higher than, or lower than the flight level identified in the dispatch release? 

Same Higher Lower 

25. If you typically request a flight level that is higher or lower than the flight level identified in the 
dispatch release, how much higher or lower is your typical request and why? 


Experience with Vertical Situation Displays 

26. Have you ever used a vertical situation display (VSD)? 

Yes No 

(if “yes,” proceed to question #27; if “no,” skip to question #28) 

27. How would you characterize your previous experience(s) using VSDs? 

Very Positive Somewhat Neutral Somewhat Negative Very 

Positive Positive Negative Negative 

Please explain: 


Experience with Electronic Flight Bags 

28. Have you ever used an Electronic Flight Bag (EFB)? 

Yes No 

(if “yes,” proceed to question #29; if “no,” you have completed this questionnaire) 
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29. What class(es) of EFB(s) have you used? (Please circle all that apply.) 


Class 1 EFBs [i.e., Commercial-Off-The-Shelf (COTS)-based systems, including laptop 
computers, that are fully portable] 

Class 2 EFBs (i.e., COTS-based systems that are portable, are connected to the aircraft during 
normal operations, and require an administrative control process for removal) 

Class 3 EFBs (i.e., installed equipment) 

30. How would you characterize your previous experience(s) using EFBs? 

Very Positive Somewhat Neutral Somewhat Negative Very 

Positive Positive Negative Negative 

Please explain: 


31. Please describe the functionality of the EFB application(s) you have used most frequently in the past: 
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Appendix K 

Post-Training Quiz with Answers 

1 . What are reference aircraft? 

Aircraft at intervening flight levels that could potentially block a flight level change, 
that meet ITP conditions, 

and are identified by the crew for inclusion in an ITP flight level change request. 


2. What are the ITP conditions a reference aircraft must meet? 

Same direction of flight (+/- 45 degrees), 

Qualified ADS-B data, and 
Meet ITP distance/speed criteria 


3. ITP distance/speed criteria are met when: 

ITP distance is at least 15 nautical miles, and closing groundspeed differential is no more than 

20 knots 


or 

ITP distance is at least 20 nautical miles, and closing groundspeed differential is no more than 

30 knots 


4. What is the maximum altitude change that can be made with an ITP? _4000’ 

5. What minimum performance must be maintained during an ITP flight level change? 

At least 300 feet per minute at assigned Mach number 
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6. Given ITP F/UAL123/65 in the free text field of an ITP request: 

a. Is UAL 123 ahead of you, or behind you? ahead of 

b. What is the ITP distance to UAL123? 65 nautical miles 

7. Given ITP L/DAL456/23 in the free text field of an ITP request: 

a. Is DAL456 ahead of you, or behind you? behind 

b. What is the ITP distance to DAL456? 23 nautical miles 

8. For this experiment , when deciding whether to change flight levels, you should try to get as close as 

possible to your recommended flight level. 

9. If an ITP flight level change clearance is received from ATC, what must first be done before starting the 
flight level change? 

Recheck that the reference aircraft in the clearance still meet ITP conditions. 


10. What minimum performance should be maintained during an ITP flight level change? 
At least 300 feet per minute vertical speed, at assigned Mach number. 
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1 1 . For this experiment , what should be done if minimum performance cannot be maintained during an ITP 
flight level change? 


Notify an experimenter. 


Please refer to the following figure when answering questions 12 through 18: 
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12. What is the commanded flight level (as entered on the Mode Control Panel)? 

Flight Level 330 

13. What desired flight level is selected? Flight Level 350 

14. What aircraft meet ITP conditions for reference aircraft? 

SAS398, DAL711, and AFR198 


1 5 . What is the ITP distance of DAL7 11? 32 nautical miles 

16. What is the groundspeed differential for DAL71 1? 10 knots 

17. Is SAS398 moving closer to you, or moving away? closer 

How do you know from the display? 

White triangle is filled in (and is pointing toward ownship) 


18. Please write out the two free text lines for an ITP request with DAL71 1 and SAS398 as reference 
aircraft: 

ITP F/DAL71 1/32 

L/SAS398/69 


130 





Please refer to the following figure when answering questions 19 and 20: 
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19. Is an ITP climb to Flight Level 350 possible? No Why or why not? 

MAH714 does not meet ITP distance/speed criteria, indicated by NO REF label 
in data tag and by ITP distance of only 1 1 NM (need at least 15 NM in all cases)_ 


20. What does the “NO REF” label mean in the data tag for MAH714? 

MAH714 cannot be used as a reference aircraft in an ITP request 

_ (does not meet ITP distance/speed criteria) 
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Appendix L 

Oceanic Operations ITP Checklist 


Desired flight level SELECT ON EFB 

Intermediate flight levels . . . IDENTIFY POTENTIALLY BLOCKING AIRCRAFT 
Reference aircraft (select at most 2) EVALUATE FOR ITP CRITERIA 


Same direction with qualified ADS-B data 
ITP distance/speed criteria. 


ITP Distance at least 

Closing Ground Speed Differential 
no more than 

15 NM 

20KTS 

20 NM 

30KTS 


[PNF] ITP flight level change data link request PREPARE AND SEND 

Free Text Format: ITP F/<flight id>/<distance> 

Note 1: F means you are FOLLOWING this Reference Aircraft 
Note 2: L means you are LEADING this Reference Aircraft 

Note 3: Enter 2 nd Reference Aircraft , if any , on 2 nd free text line , without ITP prefix 
Example: 


On receipt of ITP clearance: 

[PF/PNF] Reference Aircraft in 

ITP clearance 

If ITP criteria not met then reject clearance . Otherwise: 
[PF] Acknowledge/comply with ATC clearance 

• Constant Mach 

• Minimum 300 FPM vertical speed 


REASSESS ITP CRITERIA 


MONITOR 



[PNF] 

[PNF] 

[PF/PNF] 

• 
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Appendix M 

Group Debrief Questions 

1) Do you believe you could benefit from the ITP procedure during normal oceanic operations on a 
regular basis? 

2) Do you believe you could improve your operation with the EFB traffic display on some percentage 
of oceanic operations? 

3) How could the EFB display be enhanced to improve the performance of the ITP maneuver? 

4) How could the display be improved to enhance your situation awareness? 

5) Do you believe the EFB traffic display would be of use in a domestic en-route or terminal 
environment? 

6) Should the call sign of “NO REF” aircraft be removed when positioned as potentially blocking 
aircraft? 

7) Do you believe an expanded vertical only display would be useful? 

8) Should relative closure rate be displayed on all targets regardless of ITP desired altitude? 

9) Do you have any safety concerns with the ITP procedure? 

10) The controller is responsible for separation throughout the ITP maneuver; do you feel a need or 
responsibility to monitor the separation while performing the maneuver? 

11) If during the ITP maneuver, you observe your separation from the reference aircraft decrease to 8 
miles, would you take action? 4 miles? 
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